devopsy.kinsta.cloud

Tag: agile

  • Everything is DevOps

    Everything is DevOps

    It’s funny how we in IT tend to be absolutists, shoving everything into the current bucket. XML was going to eliminate programming—even though it wasn’t even particularly good at data representation. NoCode was going to eliminate programming (yeah, that’s a theme, I’ll skip the other 50 or so ‘going to end programming’ cycles we’ve been through), AI will eliminate IT, you must have a mobile app, even if no one will ever use it, my OS/browser/language is better than yours and will win—you get the idea.

    We’re currently coming to the end of the “Everything is DevOps” phase. DevOps is a great methodology that helped solve several intractable IT problems but, like everything, marketing and a ton of smart people that want to be “right” have taken it too far. Or have they?

    The thing with DevOps is that contrary to what those who try to make a separate market out of DevSecOps or SRE try to tell you, it was designed to literally encompass all of human IT activity. We can excuse the DevSecOps crowd, since DevOps’ roots had a definite “Security is an unimportant bottleneck” feel to it. The SRE crowd, on the other hand, appears to be another case of “my silo is special”. Or, better yet, a stompy marketer saying “We’re late to the DevOps game! Let’s invent something just like it, claim we do it more broadly and give it our own name!”

    Let’s look at what each does, for those of you who are skeptical. 1) break down silos. Check. 2) Monitor everything. Check. 3) Use monitoring results to improve product performance and stability. Check. 4) Design and implement products and product features. Check. There’s more, but you get the idea.

    Do you know what all of this ignores? You.

    One of the issues that our absolutism brings about is that marketers get to dictate what is important. Sometimes that is what enterprise IT needs. Far too often it is what the marketer is paid to dictate.

    I don’t care if you call it DevOps or SRE, I care that your organization feels like it is getting its money’s worth out of IT—something, I might add, that most organizations did not feel prior to DevOps. At least most of the ones I dealt with. I even had a job where they created a whole new department and hired me to run it to “get around those people in IT.” I spent as much time educating business leaders about not duplicating entire groups like storage services and InfoSec as I did building and deploying new products. So avoiding that kind of issue is near and dear to my heart. And we’re more ‘there’ than we have ever been.

    I do care about whether you call it DevSecOps or not. It should not be different. Shifting security left is the appropriate solution to the double-edged sword of security staffing shortages and increased adversary capabilities. So if your developers are getting secure coding feedback in their IDE or each time a build is kicked off, great! But call it DevOps. Say it is part of the DevOps process. Same with configuration reviews like cloud configuration management—it’s just part of the DevOps process, not a security step to be thought of differently.

    I could go on, but hopefully I’ve spurred your imagination a bit and don’t need to.

    You rock. Streamline your thinking to streamline your processes, ignore absolutism and call it macaroni for all it matters. Just keep doing what needs doing—at the most convenient place to do it. And keep developing, deploying and monitoring the systems that keep the company running.

     

  • Agile Sucks (Redux) | Plus: DevOps on Mars

    Agile Sucks (Redux) | Plus: DevOps on Mars

    Welcome to The Long View—where we peruse the news of the week and strip it to the essentials. Let’s work out what really matters.

    This week: Agile is bad, but “Wagile” is worse.
    Plus: This prod is even worse than yours. (more…)

  • Not Everything That is Necessary Adds Value

    Not Everything That is Necessary Adds Value

    Since the Lean production method for manufacturing arose in the 1930s, organizations everywhere have been seeking ways to apply it to reduce waste and increase productivity. This model has served thousands of companies well, from manufacturing and product design to software development. Yet, one difficulty that comes up when discussing or applying Lean in software development is the debate about what is and isn’t considered value-added work.

    Some people get defensive when certain activities are labeled non-value-add (NVA). Others argue that if a customer expects an activity and is willing to pay for it, it must be value-add (VA). From my perspective, software engineers, business leaders and others must accept the reality that it’s entirely possible for necessary processes to add no tangible value. We must understand the intent of Lean within these classifications. In doing so, we can get past the defensiveness and disagreements and move on to improvement.

    Let’s explore a few examples of NVA activities that are unavoidable—and therefore necessary—to a successful software development value stream:

    • Resolving important defects
    • Rework to meet client specifications (i.e. after clarifying requirements)
    • Project time tracking and billing (i.e. for time and materials projects)
    • Complying with regulatory mandates
    • Testing

    Some customers expect these activities to be performed. Software professionals generally agree that all of these NVA activities can be necessary. Yet many Lean proponents will continue to insist that they are NVA and should be minimized. So, where does that leave us?

    Distilling ‘Value’ to Its Essence

    Regardless of how important or necessary NVA activities might be, from a Lean perspective, unless an activity is adding value to the finished product, it is technically an NVA activity. Testing is an NVA activity. Defect correction is an NVA activity. Complying with regulations, although important to protecting profit margins for the company—and an activity that might incent customers to select your product over your competitor’s due to their reduced risk—is nevertheless a NVA activity.

    This is where we enter the grey area of “necessary but NVA activities that actually add a lot of value.” One prominent example is resolving defects that cannot be postponed. You’re probably shaking your head and thinking, “No way. Resolving defects increases quality which makes the finished software product more valuable to the end user. Therefore, it must be a value-add activity.”

    The answer is actually, “No, it is not.” Why? While the cost of defect resolution may be incorporated into the final price tag of software, there is no line-item charge for it. Most purchasers will not pay extra for defect resolution. They’ll feel that quality software is essential to their project and defect resolution should be included.

    Furthermore, they believe quality should be built-in— designed into the product and the process used to build it. That makes it an NVA activity. It’s necessary to the delivery of quality software but it doesn’t add value in the Lean sense of the term.

    The truth, whether or not business leaders want to acknowledge it, is that NVA activities are built into almost every product or service. When someone goes to pick up a prescription at the pharmacy they are never asked to pay an extra research fee. When someone test drives a car and decides to purchase it, they aren’t charged for the gas they burned during the test drive. Both of those activities were necessary for the final outcome to be satisfactory to everyone, but they added no value to the effort.

    The same is true for successful software delivery. Lean approaches form the foundation for modern software development, but being Lean doesn’t mean that you have eliminated all NVA activities. In fact, quite the contrary.

    Lean suggests that perfection is never achieved. A Lean organization values continuous improvement and is always looking for ways to reduce NVA activities. For all software projects, some elements will always be NVA. The perpetual existence of both value-add and non-value-add are facts of life.

    Optimizing Necessary Activities

    Despite the fact that these activities might be necessary and, therefore, part of the project cost, they don’t have to be excessive. Responsible software professionals and the companies for whom they work will make an extra effort to reduce waste and optimize outcomes in all NVA activities.

    Reducing or eliminating waste is one of the most important tenets of Lean software engineering. Waste can be found everywhere, from too many people working on a project and ineffective communication to an excessive number of manual processes, backlog mismanagement and rework.

    Breaking the Stigma of “Non-Value Add”

    Companies must develop quality software and sell it at a price that pays for the effort. In my experience, the way to do that is by evaluating every activity. Consider whether it is VA or NVA and whether it is essential or non-essential. Then reduce or eliminate the NVA as much as possible, even if it’s essential, and optimize the VA.

    Software professionals should be ready to defend the importance of the many necessary but NVA activities that go into a quality outcome, while remaining unapologetic about calling them what they are—a process, procedure or other activity that adds no value.

  • Let Me Reiterate – Don’t Rush to Iterate

    Let Me Reiterate – Don’t Rush to Iterate

    We in the DevOps world spend an inordinate amount of time talking about tools and roles—and there are always new tools and modifications of roles to talk about.

    Do you know what we don’t discuss as an industry? The day-to-day grind of being in DevOps. Oh, you can go to forums like Reddit and have those discussions with other practitioners, but pundits don’t discuss what it’s like to actually do the work. I secretly suspect this is because most of us pundits no longer develop code in general, let alone have a working DevOps environment. I certainly don’t code as much as I used to, and I develop more than many of my peers in punditry. For the record, we do have a working DevOps environment (two, but one almost never gets spun up) at my company, but I’ll be nice and won’t get sidetracked by talking about it here.

    We’ve talked about it tangentially, but I wanted to be direct with you all. The vast majority of your organizations do not need to iterate non-stop. And the mentality that you cannot iterate too much is simply false. While the guard gates and German Shepherds that protected production from rogue developers in the past were overreactions, there should be a happy medium. Every time change is introduced in production, there is some amount of risk. And we have a ton of tools to limit that risk these days—feature flags to turn new functionality on/off, auto-run testing before the move to production, etc.—but there is still measurable risk. And “we can just roll it back” only works in small, unsegmented projects. Once you get into nested dependencies (microservices are a great example), then versioning starts to make rollbacks an issue. Knowing the interconnected relationships between what is being changed is a big deal. So it is not as simple as the ::shrug:: “Just rebuild and deploy without your changes,” crowd would have you believe.

    And to a great extent, that is why this pundit doesn’t come out and blather about iteration speeds—the decision to iterate once every six months or every time Bob checks in code is very much an organizational/application one. How fast can the organization handle a full iteration, and how often does the application require it? The answer is rarely the insane number that vendors like to throw out to show you how great they are. There are a whole lot of organization/application combinations whose rate of change is slow enough that they could honestly schedule full-on iterations and plan updates around vulnerability patches in the underlying composite pieces. The majority only need to iterate over months. Very (and I stress very) few need to iterate even daily, let alone more than a hundred times a day. And they are indeed special snowflakes with super-tweaked environments.

    It is kind of stunning to me that a portion of the rise of Agile and DevOps was to free up IT time because we were overloaded. And then once that was done, we decided to crank up the rate of change so that IT was … overloaded. A release takes DevOps time, DevOps time and security time. It takes resources to build, test and deploy. Don’t overdo it.

    Build when you need to. Deploy when it is a benefit to customers. Keep rocking it. We’re out here using your apps, or your apps are grinding behind the scenes, keeping the business running. Either way, you know better than anyone what your app and your org need. Don’t follow the “10 billion deploys a day!” crowd; push for what works. And you are the SME in that department.

  • There are Few Enough Silver Bullets

    There are Few Enough Silver Bullets

    I was working through this week’s blog this morning, and it was laser-focused on a narrow topic. I had examples of why too much of a good thing is bad, how absolutism about methodology is hurting the majority of organizations out there, and how to get past this issue to keep improving what IT does for the organization.

    And I was nearly done when I decided that there are far too many areas of IT that suffer from absolutism, so I’d offer general symptoms and steps that work to get out of any of them.

    What do I mean by ‘absolutism’? Well, it used to be—and, in a couple of areas, still is—true that people would say, “We are an X shop.” And mean it. If you were a Cisco shop in the 1990s or 2000s, that is all you bought. Competitors were not allowed; nor was consideration of competitors. This applies to methodologies as well as product categories. In fact, today it applies to various methodologies more than products.

    And it’s bad for the organization, no matter where it comes from. While with vendors, it streamlines purchasing and offers the (in)famous “single throat to choke”, it also causes inefficiencies in costs (from your perspective, they are a monopoly, and some large vendors gained infamy by trying to take advantage of that fact), and causes people to do things in the least efficient way because that’s what the chosen methodology/solution requires.

    Sometimes, we manage to do the dance and avoid “One True Way”—take the move to cloud and continuation to multi-cloud, for example. While cloud vendors do not offer customers much leeway in pricing or priority (we customers are all cattle, not kittens), multi-cloud still offers us the opportunity to use the cheapest or best-suited cloud vendor without much drama.

    Many times, we are not so effective at avoiding the idea of a silver bullet. Take Agile. If there was a One True Ring of this decade in IT, I’d want it to be containers, but it would actually be Agile. Most IT departments zealously guard that Agile is the One True Way. Early-stage new product development is astoundingly well suited to Agile, and we needed it—if just for that part. Late-stage product maintenance, where changes are not feature-laden monuments to systems complexity, are also really well-suited to Agile and we should be happy to have it. In between? Large updates that can span data centers—or even continents—with a geographically dispersed workforce and hundreds (or more) of hours of development to get one sub-feature completed? Not really a great fit for Agile, and while many of us have said that for years, here is hoping things are changing. As I’ve said before, as maligned as waterfall was, it did the job for decades and handles large complex projects well. Maybe we need a replacement more in line with the times; one that incorporates what we’ve learned from Agile. I’d love that, actually. But what we don’t need is “We’re an Agile shop.” It is less useful than when it was said about products (at least if you only chose a single product, you got single sourcing and streamlined purchasing. With methodologies, you don’t even get that).

    Note: Agile is an example, not the point of the post. Avoid “One True Way” across your organization. Root it out and examine it. Sometimes, a single solution with a single specialized skillset is the perfect answer—at least for now. Far too often, “We are an X shop” is someone’s preference codified into corporate policy.

    To get past One True Way, you have to be there showing its weaknesses to management. There is some skill overlap when allowing multiple approaches or products, but if management starts prattling about that, simply point out the number of development and/or CI/CD environments the org is currently supporting. To show the overhead, track hours spent. Staff shortages have been endemic to IT for years. Make certain you know how much time is spent in overhead for the product or process, and try to map that to what could have been achieved with that time. It’s not perfect, but it will be illustrative. And don’t stop. Do your work, but point out every time that One True Way makes the job harder or more expensive.

    You are nailing it. Don’t let things like Agile keep you from churning out great apps for the company. I’ve been on teams where the time spent in process overhead was dragging down the ability to deliver quality. Fight that. Keep kicking it, and use the best tools/processes for the job.

  • Update Those Ops Tools, Too

    Update Those Ops Tools, Too

    We’re all pretty well settled on the idea that the rate of change in IT has been on the increase over time. From the advent of VMs to Agile to DevOps to containers, the movement has been faster—and the same is true for internal development.

    I’m going to approach this topic from three perspectives instead of my usual two. Kind of like breaking the fourth wall in cinema. We’ll discuss this movement just a bit from the perspectives of my two preferred bits, DevOps and DevOps. But for this blog, we’ll also mention DevOps—the infrastructure, tools and processes that enable the other two.

    For DevOps, languages—both updates to those currently in use and the addition of new ones—bring with them a nearly automatic update to tools used. Favorite libraries/modules, IDEs, build tools—these are all looked at and updated more frequently today than at any point in history, by my estimation. For teams that do not change languages or libraries much, source code analysis (SCA) scanning is increasingly helping to take a closer look at all those included tools and filter them based upon impact (normally security impact, but other impacts on occasion, too—like project activity and stability).

    For DevOps, the toolchain is still new enough that evolution is causing regular change, and that change is bringing with it new tools and utilities to solve the problems of this existing infrastructure. In fact, I’ve long thought the DevOps part needed a bit of a cooldown on the change and new tools/utilities front, personally.

    And finally, DevOps has gone a bit sideways in this regard. Every person—whether you call them ops, admin or DevOps—responsible for troubleshooting machines and apps has a collection of tools. But the divergence of platforms has created a divergence of tools. It is possible to work without a dedicated set of tools—and, indeed, many DevOps engineers do. Others have a folder. Meanwhile, there is another dedicated set of DevOps engineers that have a PCMCIA or USB full of useful tools that is always with them. Some have both. Of course, what tools are included in all of this varies widely also. It is an interesting artifact of our growth in IT that developers tend to share or borrow the best libraries/modules/APIs across an organization, but DevOps is more independent. As long as DevOps team members are getting the job done, no one is all that worried about which tools they used to troubleshoot. They do tend to share if asked, but with little physical evidence of what tools they used, it doesn’t come up often.

    So, what am I getting at? Look for ways to enable DevOps tool sharing on an ongoing (or at least regular) basis. An internal social media channel dedicated to the vendor your teams use to communicate, brown bags, a maintained rating system “Why I think fdisk rocks” type of thing, etc. The more power there is at the fingertips of every DevOps member you have, the better. Given our history, building an “approved” list is likely a bad idea, but having that central communication location that security can also peruse to look for red flags is not a bad idea. Of course, they are likely too busy, but worst case, such conversations can help in forensic analysis later. Not being cynical, just living IT in the 21st century.

    And keep kicking it. More and better tools will help keep all those sweet apps and servers clipping along, so there is really only upside here. Give your teams the tools and revel in your considerable uptime.

  • Defining the Next Cycle of Technology

    Defining the Next Cycle of Technology

    We’re in the middle of what we at ScyllaDB dubbed The Next Tech Cycle. Not “The World of Tomorrow” nor “The Shape of Things to Come” nor “The Wave of the Future”—because it’s already here today. We’re right in the middle of it. It’s a wave that’s carrying us forward, but it began with trends that got their start earlier this century.

    Everything is co-evolving, simultaneously, from the hardware you run on to the languages and operating systems you work with to the operating methodologies you use day-to-day. All of those familiar technologies and business models are themselves undergoing revolutionary change.

    This Next Tech Cycle Goes far Beyond “Big Data”

    We’re talking about huge data. Welcome to the Zettabyte Era. This era, depending on who is defining it and what it is measuring, either started in 2010 (if you’re measuring total data stored on earth) or in 2016 (if you’re measuring total internet protocol traffic in a year).

    Right now, individual data-intensive corporations are generating information at the rate of petabytes per day and storing exabytes in total. There are some prognosticators who believe we’ll see humanity, our computing systems and our IoT-enabled machinery generating half a zettabyte of data per day by 2025.

    Yet, conversely, we’re also seeing the importance of small data. Look at the genomics revolution.

    Because the RNA gene sequence of, say, COVID-19 is actually not that big, data-wise—you can store it in less than 100kb; less than an ancient floppy. But it is increasingly important to understand every single byte of that information because vaccinating against this global pandemic requires understanding every change of that rapidly-evolving pathogen.

    So, this next tech cycle has to scale everything from huge data and small data systems. And the database you will use and the data analytics you perform need to align with the volume, variety and velocity of the data you have under management.

    “Great! Now Make it All Multi-Cloud!”

    Also, this next tech cycle is not just the “cloud computing cycle.” AWS launched in 2006. Google Cloud was launched in 2008. And Microsoft Azure was formally launched in 2010. So, we’re already well over a decade past the dawn of the public cloud. Yet this next tech cycle definitely builds on the ecosystems, methodologies and technologies these hyperscalers provide.

    The database you use also has to align with where you need to deploy it. Does it only work in the cloud, or can it be deployed on-premises far behind your firewall? Does it just work with one cloud vendor or is it deployable to any of them? Or all of them simultaneously? These are important questions.

    Just as we do not want to be locked into old ways of thinking and doing, the industry does not want to be locked into any one technology provider.

    If you’ve just been mastering the art of running stateful distributed databases on a single cloud using Kubernetes, that’s not good enough. Now you’re being asked to do it all over again; this time in a hybrid or multi-cloud environment using Anthos, OpenShift, Tanzu, EKS Anywhere or Azure Arc.

    Computing Beyond Moore’s Law

    It’s also not the basic broadband or wireless internet revolutions. We’re a full two decades into both of those. Yet the advent of gigabit broadband and the new diverse range of 5G services—also capable of scaling to a gigabit—enable incredible new opportunities in real-time data streaming services, IoT and more.

    So how does your database work when you need to connect to systems far and near? How important are the limitations of the speed of light to your latencies? How well do you deal with data ingested from hundreds of millions of endpoints at gigabit-per-second scale?

    And finally, underpinning all of this are the raw capabilities of silicon, summed up by the transistor and core counts of current generation CPUs. We’ve already reached 64-core CPUs. The next generation(s) will double that, to the point where a single CPU will have more than 100 processors. Fill a rack-based high-performance computer with those and you can easily get into thousands of cores per server.

    And all of this is just traditional CPU-based computing. You also have GPU advancements that are powering the world of distributed ledger technologies like blockchain. Plus, all of this is happening concurrently as IBM plans to deliver a 1,000 qubit quantum computer in 2023 and Google plans to deliver a computer with one million qubits by 2029.

    This next tech cycle is powered by all of these fundamentally revolutionary capabilities. It’s what’s enabling real-time full streaming data from anyone to anywhere. And this is just the infrastructure.

    If you dive deeper into that infrastructure, you know that each of the hardware architecture bottleneck points is undergoing its own revolution.

    We’ve already seen CPU densities growing. Yet vanilla standalone CPUs themselves are also giving way to full systems on a chip (or SoCs).

    And while they’ve been used in high-performance computing in the past, expect to see commonly-available server systems with greater than 1,000 CPUs. These will be the workhorses —or, more specifically, the warhorses—of this next generation: Huge beasts capable of carrying mighty workloads.

    Memory, another classic bottleneck, is getting a huge boost from DDR5 today and DDR6 in just a few years. Densities are going up, so you can expect to see warhorse systems with a full terabyte of RAM. These and larger scales are going to be increasingly common—and, for businesses, increasingly affordable.

    Storage is also seeing its own revolution with the recently-approved NVMe base and transport specifications, which will enable much easier implementation of NVMe over fabrics.

    Now, software will have to play catch-up to these capabilities. Just as it took time for kernels and then applications inside of a vertically scaled box to be made async everywhere, shared-nothing, sharded-per-core, and NUMA-aware, this next tech cycle is going to require systems to adapt to whole new methodologies of getting the most from these new hardware capabilities. We will need to reconsider many foundational software assumptions.

    Evolving Methodologies: Agile and Beyond

    Speaking of methodologies, just look at these from the dawn of the millennium and onward. As an industry, we’ve moved from batch operations and monolithic upgrades performed with multi-hour windows of downtime on the weekend to a world of streaming data and continuous software delivery performed 24 × 7 × 365 with zero downtime ever.

    And by migrating to the cloud (and this always-on world) we’ve exposed ourselves and our organizations to a world of random chaos and security threats. We now have to operate fleets of servers autonomously and orchestrate them across on-premises, edge and multiple public cloud vendor environments.

    While Scrum has been around since the 1980s, and continuous integration since 1991, in this century, the twelve principles of the Agile Manifesto in 2001 altered the very philosophy—never mind the methodologies—underlying the way software is developed.

    The Agile Manifesto’s very first line talks about the highest priority being “to satisfy the customer through early and continuous delivery of valuable software.”

    However, the specific term continuous delivery (CD), as we know it today, didn’t take hold until 2009. It was then joined to continuous integration and coincided with the birth of what we now know as DevOps.

    With that, you had a framework for defining change-oriented processes and software life cycles through a responsive developer culture that now, a decade or two into this revolution, everyone takes as a given.

    This graph shows how continuous integration (CI) and continuous delivery (CD) evolved independently. They were eventually conjoined by the term CI/CD, whose popularity as a search term only began to rise around 2016 and did not displace the two separate terms until early in 2020. It is now increasingly rare to refer to CI or CD separately, according to Google Trends.

    Onto that baseline were built tools and systems and philosophies that extended those fundamental principles. The Chaos Monkeys of the world, as well as the pentesters, want to break your system—or break into them—to uncover flaws and defects long before something stochastic and catastrophic or someone maliciously does it for you.

    Cloud-native technologies like Kubernetes and single-source-of-truth for infrastructure methodologies like GitOps were created out of sheer necessity to scale systems to the hundreds of thousands of production software deployments under management.

    DevSecOps

    And that’s still not enough. We’ve already seen software supply chain attacks like SolarWinds, low-level system attacks like Spectre, Meltdown and Zombieload, or human-factor threats like viral deep fakes and millions of fictitious social media accounts using profile images generated by generative adversarial networks, never mind millions of IoT-enabled devices being nefariously harnessed for distributed denial-of-service (DDoS) botnet attacks. Just recently, news broke of the Log4j vulnerability—our best wishes to anyone who was patching code this past holiday season.

    These are all just the bowshocks of what’s still to come.

    So now, your AI-powered security systems are locked in combat every day in real-time against the threat actors attempting to undermine your normal operations. We know this because a growing number of intrusion prevention and malware analysis systems are built at terabyte-and-beyond scale using distributed databases as their underlying storage engines.

    Hence, these days, rather than just DevOps, increasingly we talk about DevSecOps—because security cannot be an afterthought. Not even for your MVP. Not in 2022.

    And these methodologies are continuing to evolve.

    Summary

    This next tech cycle is already upon us. We’re in the thick of it. You can feel it in the same way you’d yearn for major life or career change. Maybe it’s a programming language rebase. Is it time to rewrite some of your core code as Rust? Or maybe it’s in the way you are considering repatriating certain cloud workloads or extending your favorite cloud services on-premises such as through an AWS Outpost. Or maybe you’re actually looking to move data to the edge? Planning for a massive fan-out? Whatever this sea change means to you, you’ll need rock-solid infrastructure that’s available today to take workloads into production right now, but flexible enough to keep growing with your emerging, evolving requirements.

  • Wi-Fi 7 Chips Ahoy | Google ‘Gone Downhill Fast’ | Real-World ‘Severance’

    Wi-Fi 7 Chips Ahoy | Google ‘Gone Downhill Fast’ | Real-World ‘Severance’

    Welcome to The Long View—where we peruse the news of the week and strip it to the essentials. Let’s work out what really matters.

    This week: The first silicon for Wi-Fi 7, Google’s search quality “is” failing, and what Severance tells us about The Great Resignation. (more…)

  • Agile Development at 20: Can it Thrive in the Hybrid Work Era?

    Agile Development at 20: Can it Thrive in the Hybrid Work Era?

    The Agile Manifesto turns 20 this year, and while many of the core tenets are still applicable, the world has changed considerably in the last two decades. What would the Agile Manifesto look like if it was written today and took into consideration our new reality of hybrid working environments? Changes are most definitely needed, as software leaders still find themselves struggling to navigate the murky waters of 21st-century project management to get the most out of their team.

    Developers have many complaints about Agile, but most of them boil down to the fact that they often spend more time planning for work than doing it. For example, developers sometimes feel that frameworks like Scrum are top-heavy and bureaucratic, which is the opposite of their intended purpose. Developers can also feel disconnected from the bigger goals of their project, being trapped in an endless list of “to-dos” without the time to think holistically about the features they’re creating.

    With these concerns in mind, let’s take a quick look at what the original Agile Manifesto actually included. It started with four primary values:

    • Individuals and interactions over processes and tools
    • Working software over comprehensive documentation
    • Customer collaboration over contract negotiation
    • Responding to change by following a plan

    On their faces, all of these seem reasonable; even approaching plain ol’ common sense. But in their zeal to focus on code development rather than bureaucracy, the authors couldn’t have anticipated how much the industry would change. Consider some of the trends we’re seeing in today’s software development teams.

    First and foremost, the idea that “every company will be a software company” has become a reality. It’s hard to imagine an industry that doesn’t leverage software in some way to address its core competency. This trend has, in turn, fueled another major trend: The war for developer talent. Skilled developers were already at a premium prior to the Great Resignation in the wake of the pandemic. Now, developers hold substantial leverage over their employers in ways never seen before.

    The Pandemic and the New Normal of Hybrid Teams

    The pandemic has also brought us the new normal of remote, distributed teams and asynchronous work styles and skilled workers are spread across the globe. There are also new technologies that enable more automation for software teams, including machine learning and AI and other methodologies (like DevOps and DevSecOps) that reduce workloads for developer teams by using automation and process-driven workflows.

    These trends have given rise to particular needs for almost all hybrid developer teams. They include significantly improved productivity for faster software delivery and developer-friendly tool experiences that speak directly to how developers work today. Teams also require guided, asynchronous processes that map to the reality of highly distributed teams. Finally, as the cloud era is firmly a reality, developers need new technologies that offer the ability to scale predictably without suffering from increased complexity.

    Now, let’s look at the original Agile values through the lens of today’s trends and needs. The first is valuing individuals and interactions over processes and tools—this definitely requires an update for today’s developer teams. Specifically, process is a prerequisite for productivity. It is an enormous challenge to measure productivity or determine if the team is improving without a consistent process to follow. Without agreed-upon rules for working together, software development can quickly devolve into chaos. Second, putting individuals and interactions over tools seems like a restrictive viewpoint today. Good tools are explicitly designed to facilitate productive interactions between individuals. If they don’t accomplish this goal, they are poor tools; but there are many solid reasons that great tools can increase productivity and make developer teams’ lives much more manageable.

    The second value of the Manifesto also seems to present a false choice, as it prioritizes working software over comprehensive documentation. Its heart is in the right place—who wouldn’t want to focus on building workable code rather than creating documentation? However, in the world of distributed teams, detailed documentation is critical for both remote and hybrid teams. This also encompasses documentation that goes beyond just the codebase. Documentation involving decisions, requirements, process, etc. also provides context for teams that are adjacent to developer teams and help them understand progress and goals. Documentation is also helpful for addressing the skills gap and talent shortage. New hires or newly-promoted team members need to get up to speed quickly and provide value as quickly as possible. In addition to training, good documentation can help them do that and help their teams be more productive more quickly.

    The third value of the Agile Manifesto said that we should focus on customer collaboration rather than contract negotiation. I believe this needs a slight but essential adjustment for today’s hybrid teams. Specifically, teams should be building with end-users,—not just “customers”—in mind, which means that the needs of whoever will be using the end application regularly should be paramount. This is particularly important today given how many end users are involved in buying decisions.

    I think the final value of the Agile Manifesto—responding to change over following a plan—needs no edits whatsoever. This remains great guidance for developer teams everywhere, and it seems the Manifesto should take this advice and adapt to the many changes that have occurred since it was first written.

    Many of the 12 principles of the Agile Manifesto continue to be outstanding guidance that is applicable for most developer teams. There is, however, some nuance that should be considered for a couple of them. To wit:

    “The most efficient and effective method of conveying information to and within a development team is face-to-face conversation.”—This is true, but today, remote and hybrid teams are a reality. That’s not changing any time soon. Teams need to optimize for asynchronous communication and collaboration.

    “Agile processes promote sustainable development. The sponsors, developers and users should be able to maintain a constant pace indefinitely.”—Sure, but given the myriad demands on software teams, developer burnout needs to be monitored and managed or overall quality and productivity will suffer.

    “Simplicity—the art of maximizing the amount of work not done is essential.”—I couldn’t agree more, but in my mind, this means that teams should leverage automation where possible to maximize “work not done.”

    Conclusions

    In general, it seems that the Agile Manifesto holds up pretty well, provided that today’s workplace realities are taken into account. If software teams recognize that tools and processes create conditions for greater productivity and that documentation is critical in a remote/hybrid world, their remote Agile teams will be much better off. In addition, teams should build with end-users in mind, not just customers. With this added nuance and recognition of a changing world, today’s software teams can continue to thrive while leveraging this decades-old advice.

  • Your DevOps Teams Can Do Better. Here’s How.

    Your DevOps Teams Can Do Better. Here’s How.

    As more organizations focus on digital transformation as the centerpiece of their growth strategy, software developers and IT teams are under heightened pressure to perform—quickly and efficiently.

    That begs two questions: Can we do better? And if so, how can we do better?

    The answers can be elusive, especially for DevOps teams that have over-relied on processes and models that, while previously successful, may not be as relevant for today’s projects. Of course, the point about relevance has been further magnified during the pandemic, when sudden shifts to remote working left many teams scrambling to ensure they had the capacity and infrastructure to support massive work-from-home cohorts.

    Software development and IT operations support teams that are winning by doing better in this new era are truly embracing Agile. And that means taking to heart the definition of “Agile,” which is a mindset of values and principles to continually evolve your way of working (WoW) to ensure your productivity and value are tailored to fit the tasks at hand.

    Conversely, teams that have relied on their go-to framework or approach for every assignment are often missing the boat with respect to efficiencies, effectiveness and even ROI. This is because they aren’t assessing whether their proven WoW is useful for their current projects.

    That can lead to underwhelming results. A report from Digital.ai revealed that while over 90% of software development teams take an Agile approach to their work, a whopping 96% of their digital transformation projects have failed or been delayed. Clearly, that suggests Agile has worked for them in the past, but the nature of digital transformation projects is calling for teams to adapt to new or hybrid approaches that may differ from their go-to WoW.

    Challenging? Yes. Doable? Also Yes

    While there is no quick fix to attain true flexibility amid complexity, the journey begins with the recognition that, yes, you can do better. And to do better—to become an adaptive, lean, responsive and learning organization under any circumstances—there are a number of steps to take to ensure your journey to Agile freedom is successful.

    Here are five that I believe must be considered for DevOps teams to become better and more effective:

    • Consider all options. Agile has been practiced for more than two decades, and literally hundreds of frameworks and approaches have been developed and implemented. If you examine your WoW in a context-driven, pragmatic manner you are more likely to select the best-fitting technique given the situation.
    • Do your homework. Your team leaders may not be familiar with the breadth of the Agile, traditional and hybrid frameworks at their disposal. But there’s help available. Seek out online courses and certifications designed to help organizations and individuals explore and navigate the multitude of Agile frameworks that they may currently be overlooking.
    • Focus on value. It’s great if your gut tells you that your Agile framework is reaping real results, but can you prove it? If your senior leadership has invested in Agile training, coaching and certifications, they’ll demand to see some tangible returns. With well-chosen Agile approaches that match the work required, you will be able to deliver the proof that Agile is driving measurable value that matters. Even further, learning which metrics to use in different situations can help measure this value.
    • Sidestep bottlenecks. By choosing the best WoW for the project at hand, you are more likely to eliminate the wasteful or inefficient processes that can add hours or days to any assignment. Understanding how to unclog these pipes is critical in improving the flow of value to your customers.
    • Motivate with morale. If your team is frustrated and morale is low, productivity, the quality of the work and your retention rate will be, too. But, if you’re able to do Agile well, this can help your teams be happier, more productive and more successful in accomplishing project goals. Ultimately, that means a more joyful customer base, too.

    Conclusion

    In the long run, the framework or approach you choose isn’t as important as your willingness to be open-minded in applying Agile frameworks to projects on a case-by-case basis. As you become more comfortable with new or different approaches, you will promote continuous improvement through carefully considered experimentation.

    That means you’ll become a learning organization, one that is open to change and evolution. With that mindset, your development and IT ops team will do better—guaranteed. Because true agility comes from freedom, not frameworks.