devopsy.kinsta.cloud

Tag: technical debt

  • Enterprises Face Growing Technical Debt

    Enterprises Face Growing Technical Debt

    Technologists are always espousing new, trendy development solutions. More robust frameworks, increased delivery, better integration. But rarely do these proponents consider the long-term side effects of introducing new technologies. As a result, technical debt is an all-to-common occurrence in most companies.

    Nearly a third of IT budgets are dedicated to addressing technical debt. Devoting so much effort to maintaining legacy code leaves fewer resources to support the status quo, let alone develop new solutions. In fact, a recent study found that 69% of IT leaders identify technical debt as a significant threat to their companies’ ability to innovate.

    “The Growing Threat of Technical Debt,” a report from OutSystems, surveyed 500 IT leaders about this conundrum. Below, we’ll explore the top causes of technical debt and consider ways to prevent it from getting out of control.

    What is Technical Debt?

    Technical debt seems counterintuitive. Isn’t technology supposed to help organizations become more productive? Why would it be a burden? Well, technical debt often arises from hasty decisions taken in the short term that have long-term consequences. It’s also often the result of a company juggling many competing technologies, some of which have become outdated.

    Technical debt can carry a hefty price tag. But this debt isn’t so much measured in the cost of hosting legacy technology or running broken code, but in the hours lost maintaining and patching it. The report found that, on average, businesses devote 28% of their IT budgets to addressing technical debt. This number increases to 40% within large enterprises.

    Common Causes

    There is not one single factor causing technical debt. Many issues within an organization contribute to the problem, including reliance on stale technology, pressure to deliver in the short term, constant change, developer churn and incorrect architectural decisions.

    The top cause of technical debt is too many development languages and frameworks, cited by 52% of respondents as a big or critical problem. Legacy technology can weigh an IT department down. But, it’s not necessarily only old tech getting in the way—it could be that IT is supporting too many competing agendas.

    The second top cause is a high turnover rate within developer teams. In today’s competitive climate, quality engineers are in short supply, and hiring can be challenging. It is thus difficult to attract and nourish steady engineering talent. If developers frequently leave for greener pastures, especially before documenting their procedures, best practices can easily be lost and efficient use of technology is stunted.

    The study found that other common causes include accepting known defects to meet deadlines, using outdated programming languages and frameworks and dealing with challenges in serving new markets or segments. At times, teams release software with known defects simply to meet deadlines. The sheer size of a large organization can lead to increased technical debt, and not addressing it compounds the issue further.

    In terms of industries, the report found the health care and social assistance category hit hardest. In this sector, 35% of IT budgets go toward addressing technical debt, on average. The next most-burdened group is the financial category, which was at 29% according to the report.

    The Costs of Technical Debt

    What are the costs of technical debt? There’s a high price to maintain, reduced hours for innovation and a detrimental effect on overall software performance. Team dynamics could also incur adverse reactions from sustaining technical debt.

    Technical debt requires an disproportionately high amount of time to maintain. On average, engineers spend 33% of their time dealing with technical debt, found Stripe’s “The Developer Coefficient” study. This equates to about 13.5 hours of an average work week.

    The more debt you take on, the smaller the budget left for running status quo operations and building new capabilities. Supporting technical debt not only has financial implications but could stifle future prospects; in fact, 69% of respondents said technical debt limits their capacity to innovate. In today’s acceleration economy, not setting aside resources for digital innovation, could break a business.

    “The combination of old code along with the new generation of mobile apps, stack applications and SaaS sprawl are robbing organizations of resources, time and the ability to innovate,” said Paulo Rosado, CEO, and Founder of OutSystems.

    Technical debt could also stunt performance. According to the survey, 61% of respondents said it causes a significant drag on their company’s performance. Stale projects could interfere with streamlining customer experiences and digital transformation goals.

    Lastly, technical debt could also negatively affect morale. Few engineers take pride in repeating inefficient processes or working with obsolete tech.

    Alleviating Technical Debt: Be Intentional

    Similar to financial debt, technical debt comes with compounding interest. This interest could grow exponentially if left unchecked. So, what steps can organizations take to reduce their technical debt?

    To help alleviate the problem, the report recommends reversing the top two causes of technical debt by stemming turnover in development teams and limiting the number of new programming languages and frameworks you adopt.

    Other methods involve taking stock of your holdings to understand the presence and impact of legacy technologies. Next, rethink and redirect your strategy from simply maintaining legacy dependencies to addressing them and replacing them with modern styles.

    “A lot of bad tech debt comes from building too much and getting stuck spending more time on maintenance and bug fixes than expected,” said Jon Thornton on the Squarespace blog. ”The key is to be intentional about what you invest time in and aware of the costs you’re taking on.”

    Of course, enterprise IT must take on some form of technical debt to function—it’s futile to get rid of it entirely. Therefore, as Thornton advises, be intentional about it and stay aware of the costs you take on. When adopting new technologies, it’s best to consider maintenance implications and plan with longevity in mind.

    For more insights, download “The Growing Threat of Technical Debt” report in full here.

  • Cattle, Not Kittens – Revisited

    Cattle, Not Kittens – Revisited

    The early days of DevOps were filled with pronouncements that we needed to treat our servers as cattle, not pets (kittens). For a year or so, the refrain was everywhere. And we did. We detached servers from dedicated hardware, we made spin up and spin down so easy that we just replaced malfunctioning servers (which has caused its own problems), we have even made servers designed to spin up, do one thing well, then spin back down until needed again – something unthinkable even a decade ago.

    And we’ve reaped benefits. Servers suck up less attention from the DevOPS side, and standardization has made it easier to code for them, moving things along and allowing members of another team to pitch in when there is a crunch because of familiarity.

    But IT is filled with people, and something happened in the transition. In hindsight, we should have seen it as a possibility, but things changed so much, so fast that I don’t think anyone was considering this possibility.

    Our DevOps tools/toolchains/processes became our pets (kittens).

    Now the special snowflake that requires specific knowledge and has to be treated with kid gloves is the DevOps step, not the servers it uses. Like pet servers, a fragile step that is working is avoided like the plague, when it should be shored up. Like pet servers, a process that is locked into a specific tool, OS, or platform is left that way, even when other options are a better solution. Like pet servers, the person who “knows all about” that is left to manage the step that should, by all rights, be easily understandable by anyone on the DevOps team.

    It’s time to start treating DevOps like cattle, not kittens. Some steps (like some servers) will be more difficult to make less pet-like and more cattle-like, but if history is any gauge, the payoff is worth the investment.

    We don’t have a cross-toolset language, other than doing things in the various shells and calling the shell scripts from a given toolset (not the best solution, just an example of where we’re at), but that shouldn’t stop us from standardizing and genericizing as much as possible. Less fragility, more reusability and a broader support team is good for everyone.

    You all are knocking it out of the park. We – all of us – are spending more time at work and at home in the digital space than ever before, and, generally speaking, the tools “just work.” That’s all on you. Keep kicking it. But standardizing to keep things running smoothly just makes sense. You know your org; it is possible that the easiest route to cattle is to standardize on a single toolchain (perhaps one or two toolchains with reusability documented) possibly to shove a bunch of steps into scripts and share amongst whatever tools each team chooses (I, and history, strongly discourage this approach, but neither I, nor history, has to live in your environment. There might be a good reason for this, so I include it here).

    All I’m saying is, don’t re-implement every step over and over if you can avoid it, and don’t make steps so arcane that you need a specialist (not internal, and definitely not external). Keep rocking it, and get those cattle moving!

  • Playing Catch Up

    Playing Catch Up

  • Value Stream Management: Treat Your Pipeline as Your Most Important Product

    Value Stream Management: Treat Your Pipeline as Your Most Important Product

    When it comes to accelerating customer value, it’s dangerous to focus only on the development and delivery of a digital product. For many enterprises, most time is invested before a single line of code has been written. That explains the recent focus on value stream management (VSM)—it’s forcing us to think beyond Agile and DevOps and consider the whole system, not just one stage in a complex process.

    By extending our visibility further upstream where work originates, we can begin measuring the value of creating and protect workflows end-to-end to meet business outcomes sooner. Thinking about how software delivery work moves between all teams in the value stream from request to operation leads us to the significant realization that our tooling pipeline is our most important product and we should treat it as such.

    The Rise of Value Stream Management

    The growth of value stream management in software delivery has emerged because Agile and DevOps are not enough to remain competitive against the breakneck speeds of the tech oligopoly. The progressively maturing practice takes a systematic approach to continuously measuring and improving end-to-end flow to accelerate time-to-value. Consequently, new tooling is emerging as we think end-to-end in how we collaborate and work faster together—and measure our productivity and effectiveness.

    Measuring and understanding end-to-end flow isn’t easy. Software delivery is highly technical and creative knowledge work that can span thousands (or tens of thousands) of IT practitioners. Information is scattered across heterogenous toolchains comprising discipline-specific tools. In fact, some 42% of large-scale software delivery organizations use four to 10 core collaboration tools for portfolio management, design, enterprise agile planning, testing and ITSM, and dozens of additional tools to support the CI/CD pipeline and operations. 

    Mik Kersten, “Mining the Ground Truth of Enterprise Toolchains, IEEE (Volume 35, Issue 3, May/June 2018, p12-17

    Measuring flow, therefore, involves tracing constituent units of value creation and protection as they traverse all team tooling in the form of artifacts such as activities, capabilities, epics, features, stories, bugs and incidents. Yet for too long we’ve been too focused on “the right side” of the value stream, thinking about how products are built and delivered through Agile and DevOps tools, without visibility on how they’re requested, analyzed, processed and designed. Our ability to support customers is hindered because we don’t know where and why work is slowing down outside of development. Our product pipeline is broken, our visibility fractured.

    A ‘Broken’ Product Pipeline

    Traditionally, CI/CD tools include source code management, build, deploy, quality management and service management. Moving more to the right, you have release management and service management. But too often organizations have ignored integrating tools further to the left such as planning and portfolio management, product management, requirements management and risk management (static and dynamic security scanning tools). This impacts our ability to see and measure end-to-end flow.

    In”Accelerate,” lead time is defined as the portion of the value stream from (last) code commit until deployment. While this is essential to improve, it only covers the time from the build being produced through deployment. We need to account for the time that elapses before it hits a developer’s backlog. To shift IT from a project to a product focus, you need to reduce end-to-end flow time by measuring earlier in the delivery life cycle by “shifting left.” As such, you need tooling across the whole value stream through the ideate, create, release and operate phases. A focus only on optimizing the CI/CD tooling and practices is too narrow a view that produces diminishing returns over time.

    The Great Disconnect Between IT and the Business

    Because of this lack of a holistic view, IT teams don’t feel like they are able to influence the business side of the house, where most of the work in progress (WIP) starts, such as requests for features or application changes. These intake processes many times result in teams taking on too much WIP and neglecting work. Many teams need to “stop starting” and to “start finishing” to deliver business value more quickly. Crucially, this is the area where business leaders have their own processes and where tools are less likely to be used; it’s a world dominated by spreadsheets, documents and PowerPoint presentations. This is where work is least visible and measurable. And this is precisely where we need VSM: to turn the light on starting with the customer needs so we can continuously improve end-to-end flow. This requires a culture shift from both IT and business/product owners. And that requires taking a broader, more holistic view of the tools and practices in the business space.

    Architecting for Flow

    An end-to-end integrated pipeline product can be architected for flow. For example, tools used for visual system management (e.g. Jira, Azure DevOps) are becoming more of a commodity with the real differentiation becoming how they can be integrated with tools used for portfolio, defect and service management, since those are the sources of work that will flow into product-based teams for development of business value and unplanned work. Teams should think about how they can work more closely with business portfolio teams on continuous flow (and funding) because eventually this work will become the next bottleneck that needs to be addressed on the journey to accelerate business delivery capability.

    Keep an Eye on Value Protection Work 

    With end-to-end visibility into work across the pipeline, you can abstract that connected technical data into overarching value stream metrics—known as flow metrics—to identify areas where to run experiments to improve end-to-end flow. Here we mustn’t just focus on work that creates value but protects it, too.

    We should provide our software delivery teams with the psychological safety to devote time to address debt and neglected work. Debt creates a drag on feature velocity because changes become more complex and take longer to make. In addition, with higher technical debt, more time is spent on addressing defects, which starves the capacity to perform feature work. As debt builds, the following is observed:

    • Flow velocity decreases.
    • Flow time increases.
    • Quality decreases as the code base becomes more fragile/costs increase.

    All of these result in reduced happiness for both the customer and the team, increasing the risk of team burnout. There are many cautionary tales of this occurring where market leaders ended up losing considerable market share due to inattention to technical debt. These stories exemplify why it is necessary to observe flow distribution to ensure that prioritization decisions made with the product owners, include providing necessary capacity to address technical debt.

    Applying the concepts of value stream architecture to your end-to-end pipeline is the key differentiator for organizations that are trying to make it through disruptive periods such as today, as COVID-19 widens the gap between those who have mastered software at scale and are thriving and the many enterprises struggling to pivot and adapt in the digital-first climate.

  • Documenting: Don’t Build Your Replacement’s Nightmare

    Documenting: Don’t Build Your Replacement’s Nightmare

    Want to reduce headaches tomorrow? Document everything today

    About once a year I feel the need to remind you that you are creating technical debt. DevOps is good stuff, and most shops got over the “whatever a given project team decides” multiplication of services pretty quickly, but every line of DevOps code you write is creating technical debt.

    There is no way around this fact. You are writing code (scripts, if you prefer) and developing configs that make assumptions. Those assumptions will change—products will get incompatible upgrades, vendors will go under, libraries will become outdated … the list goes on.

    We have a long history with technical debt. DevOps partially grew out of the need to spend less time maintaining and more time releasing features, and yet we’re re-creating technical debt; in some cases at a much larger scale. It is unavoidable but can be managed. Keep a reference list of assumptions—“This app assumes environment contains X,” or “This app supports Cisco Network APIs version X.Y”—so you know where to look when things inevitably go wrong. The list conveniently doubles as a handy reference to what needs updating when you find out your middleware vendor is going under or that MySQL is out and MariaDB is in.

    The more cutting-edge tools you use, the more likely that you are baking future problems into your environment. This is not advice to avoid cutting-edge; it is advice to recognize reality and keep closer track as you move new things into your environment that might be half-baked or whose market is destined for consolidation.

    Our IT environment has become super-complex. DevOps helps us manage that complexity but discourages adequate documentation. We’ve all heard, “It’s self-documenting” (no, it’s not) or “There is basic documentation” (basic almost always means, “We scratched some notes when it was first set up”… and haven’t touched them since). Don’t do that. Make sure that the new guy after all of you have left can figure out what is going on in this highly complex environment. It is not up to new people to be super-learners with mind-reading skills; it is up to implementors to make certain new people can figure out what is going on without wasting 5,000 hours pawing through scripts.

    It does not take much time to document dependencies. It doesn’t take much more to document relationships. Future you will be grateful when they walk in and need to come up to speed in a short amount of time. Because it doesn’t matter how automated it is when the automation breaks—it matters how quickly staff on hand can fix it and get things going smoothly again.

    You’re keeping the world running. Take a second and make sure the next person can, too. And keep rocking it.

  • Communicating with Management About Technical Debt

    Communicating with Management About Technical Debt

    We’re all familiar with the challenges of technical debt. While it’s often unavoidable–due to time constraints, resource considerations or simply a lack of good information–it still weighs on our minds and puts us on the defensive. It makes future changes that much more difficult. As it builds up, it generates that nagging sense of an unpaid bill that’s accumulated interest and is about to come due.

    But, perhaps, the most challenging aspect of technical debt is simply this: getting senior business leaders to understand what it is and convincing them to do something about it.

    There are many reasons we’re not getting through. Management is often under significant pressure to deliver results with little patience for discussing anything that doesn’t directly contribute to a specific outcome. Resources may be limited. It’s simply human nature to prefer short-term gratification–developing a new customer-facing app, for example–to the longer-term but behind-the-scenes benefit of paying down technical debt.

    If we’re honest, we also bear some responsibility for our lack of progress. We need to do a better job of clearly defining and explaining why technical debt should matter to business leaders. But, how do we do this?

    Here are five ways to explain and highlight the importance of addressing technical debt to key decision makers.

    Drop the Tech Talk

    When discussing technical debt, it’s easy to fall back on shop talk–i.e., explaining technical debt in IT terms. This is a battle that must be waged in business terms. Fortunately, there are many parallels between technical and financial debt that business leaders are familiar with.

    All healthy businesses carry financial debt on their books, and leaders know that debt must be kept within certain limits in order to maintain a healthy balance sheet. Indeed, they are acutely aware of the consequences of carrying too much debt. Not only does it endanger the company’s credit rating–making future borrowing more expensive–but it also accumulates interest and may force companies to curtail certain actions, such as paying dividends or buying back stock, that are important to shareholders. That’s why, when debt levels rise to a certain point, most companies bite the bullet and make a concerted effort to pay down debt before embarking on another business-building cycle.

    Outline the Potential Business Impact for Carrying too Much Technical Debt

    Technical debt must be kept within certain limits to avoid hurting the business. Unpaid technical debt generates interest that increases over time and adds to the cost of fixing the problem. Carrying too much technical debt leads to negative business (versus IT) consequences, including:

    • Lost sales and/or lower productivity due to system outages.
    • A loss of competitive advantage due to a progressively poorer user experience.
    • An inability to adapt to market changes.
    • Fines due to security breaches.
    • Poor decisions resulting from inconsistent or untimely data.

    Find Creative and Visual Ways to Tell the Story

    There are several different ways you can contextualize technical debt–from using everyday analogies, such as the importance of daily household upkeep, to the benefits of preventative maintenance, as in the classic Fram oil filter “pay me now or pay me later” commercials. You can also leverage compelling visuals like the one below to dramatize your arguments.

    Technical debt
    Source: Projectmanagement.com

    Rinse and Repeat

    In making your case to business leaders on the need to address technical debt, it’s important to adopt a campaign approach. Think like advertisers who measure their impact in terms of reach and frequency–how many people they reach and how many times those people are exposed to their message. While you’re not running an advertising campaign, you need to be prepared to make your case over time, and to reach both the decision makers and the people who influence the decision makers. One email, or one presentation to management, isn’t going to get your message across.

    Creating a Healthy IT Balance Sheet

    Technical debt is inevitable and, in some cases, makes sound business sense–i.e., when speed-to-market is critical, when resources are limited or information is incomplete. If technical debt reaches a certain level, it makes good business sense to forego immediate gratification projects (developing that shiny new app) in order to pay it down.

    That’s why framing technical debt in a manner that business leaders understand (i.e, through the lens of financial debt) can make business leaders more inclined to realize the importance of managing it, as they do other risks facing the business.

    Of course, responsible IT leaders not only surface issues that require management attention, they also offer potential solutions. For example, funding strategies to pay down technical debt, including ongoing efforts and small and large improvement projects.

    By treating technical debt as a business rather than a technical issue, technology officers stand a better chance of breaking through with management, paying down those nagging old debts and finally getting their IT balance sheets in order.

  • Prevent Technical Debt by Knowing When to Remove Feature Flags

    Prevent Technical Debt by Knowing When to Remove Feature Flags

    As more organizations use feature flags and experiments, it’s critical to understand that some of these changes should exist for only a short or specific period of time, and then be removed from your codebase. Removing flags when appropriate avoids technical debt and complexity that can make your code fragile, harder to test and more difficult to support.

    The simplest ways to evaluate when a feature flag or experiment should be removed include looking at how long it’s been in the system and how many different states it’s in. If the feature flag has been in your system for a long time and all of your users have the same state of the feature flag, then it should likely be removed.

    However, evaluating based on just those two factors isn’t enough. It’s smart to also evaluate the purpose of a flag or experiment before finalizing the decision to remove it. The appropriate lifetime of a feature flag or experiment, and when it should be removed from your codebase, depends heavily on its use case. 

    When to Remove Temporary Flags and Experiments

    If a feature is designed to be rolled out to everyone and you don’t expect to experiment on the feature once it’s launched, then you’ll want to ensure you have a ticket tracking its removal as soon as the feature has been fully launched to your users. These temporary flags may last weeks, months or quarters. Examples include:

    • Painted-door experiments: These experiments are intended to be used only in the early phases of the software development lifecycle and aren’t intended to be kept in the product after they have validated or invalidated the experiment hypothesis.
    • Performance experiments: These experiments are intended to put two different implementations against each other in a live, real-world performance experiment. Once enough data has been gathered to determine the more performant solution, it’s usually best to move all traffic over to the higher-performing variation and remove the related flag.
    • Large-scale refactors: When moving between frameworks, languages or implementation details, it’s useful to deploy these rather risky changes behind a feature flag so that you have confidence they won’t negatively impact your users or business. However, once the re-factor is done, you hopefully won’t go back in the other direction.
    • Product re-brands: If your business decides to change the look and feel of your product for brand purposes, it’s useful to have a gradual rollout to gracefully move to the new branding. After the new branding is established, it’s a good idea to remove the feature flag powering the switch.

    How to Review the Lifetime of Permanent Flags and Experiments

    If a feature is designed to have different states for different customers, or if you want to control its configuration for operational processes, then it’s likely the flag will stay in your product for a longer period of time. Examples of these flags and experiments include:

    • Permission flags: These flags are useful if you have different permission levels in your product, such as “read-only” that doesn’t allow edit access to the feature. They are also useful if you have modular pricing like an inexpensive “starter plan” that doesn’t have the feature, but a more costly “enterprise plan” that does have the feature.
    • Operational flags: These flags control the operational knobs of your application. For example, these flags can control whether you batch events sent from your application to minimize the number of outbound requests. You could also use them to control the number of machines that are used for horizontally scaling your application. In addition, they can be used to disable a computational expensive, non-essential service or allow for a graceful switchover from one third-party service to another in an outage.
    • Configuration-based software:  For any software or product that is powered by a configuration file, this is a great place to seamlessly insert experimentation that has a low cost to maintain and still allows endless product experimentation. For example, some companies may have their product layout powered by a config file that describes in abstract terms whether different modules are included and how they are positioned on the screen. With this architectural setup, even if you aren’t running an experiment right now, you can still enable future product experimentation.

    Note that even if a flag is meant to be permanent, it’s still critical to regularly review these flags in case they become obsolete or should be deprecated, which would then add technical debt to your codebase.

    Establishing a Feature Management Process 

    Creating an always-on feature management process goes hand-in-hand with using feature flags and experimentation effectively. Some organizations use a combination of a task tracking system and their feature flag and experiment platform to manage this cleanup process seamlessly and quickly. If the state of feature flags and experiments can be synced with a ticket-tracking tool, then an engineering manager can run queries for all feature flags and experiments whose state has not been changed in the past 30 days, and track down owners of the feature flags and experiments to evaluate their review. Other teams schedule recurring feature flag and experiment removal days to ensure engineers review the oldest items on a regular basis to minimize technical debt.

    The benefits of feature flags and experimentation to help teams ship code and launch new features quickly and safely far outweigh the needed management to minimize codebase technical debt and complexity. When managed as a recurring process, looking at all key factors (lifetime, state and use case), teams can feel confident that their systems will remain efficient and reliable.

    — Asa Schachar

  • App Mobility Is the New Use Case

    App Mobility Is the New Use Case

    Selling increased investment in DevOps is not always easy. It takes time to implement, resource away from other projects and inevitably there is cash outlay. If not done carefully, it also appears to introduce chaos, something most of us in IT strive to avoid.

    But the benefits, at this point, are easy to document. Plenty of companies have tried and succeeded in moving to a more agile/DevOps approach to deployment. Yes, companies have failed, but the publicly acknowledged failures have also been dissected for lessons learned, which can help in your transformation.

    Every advice site suggests tying DevOps/digital transformation to business goals, which is a good idea, but is harder to do most of the time.

    Not any more. The truth is the future is mobile. Not as in mobile phones or cross platform (though both have a place in the future), but in the ability to run an app where it makes the most sense today with the knowledge that tomorrow there might be a better place to run it.

    Historically, the architectural choices of yesterday’s teams become a burden on today’s teams. I’ve worked at a major insurer where a mainframe flat file DB system was keeping an application from moving. I’ve worked at a power company where we were not allowed to modify a system running on OS/2 because no one quite knew how to maintain it. I’ve worked at an energy services company where I spent a year trying to re-create Novell IPX solutions in an Ethernet environment…The list goes on.

    What if we could avoid leaving such a legacy to future teams, and be more responsive to business needs/profitability in the future? That is, in a nutshell, what containers offer you. Today, you are running your app internally in a container management system. Tomorrow you’re running it on AWS in a container management system.

    There are things to watch out for–most vendor-sponsored container management makes it very easy to use containers on their platform of choice, but difficult to move containers. IBM, as one of the least intrusive examples, offers integration with their proprietary tools. Once you’ve taken those integrations, your mobility/options are more limited. As I mentioned, IBM is one of the more open companies out there. They offer value for tying you down, but if you don’t need those services, it’s better not to use them so you can move the application at will.

    Situations change, and containers allow for those changes. You may have a huge budget for external cloud apps today, but reasons ranging from internal cost decisions to regulatory changes could force your organization to reconsider options in just a few short months.

    That makes mobility important to business. So does cost management. No matter what the cheapest architecture to deploy on tomorrow is, wherever makes the most sense is going to be a hit.

    But how portable are they? Depends upon the app, and the implementation. An application that is developed as a group, able to run in a single pod (or multiple exact copies in multiple pods) is an easy thing to move about. One with containers spread across a variety of platforms, each doing what it is best at, will be more difficult to move. Persistent storage kind of guarantees that most enterprise apps have limited mobility–at least for the time being.

    Any app that can be moved to the cloud can be containerized first, granting mobility that means no need for massive conversion efforts if the cloud doesn’t work out. Any app pulled back from the cloud to the DC has the same potential, the issue of where the data lives is real no matter what architecture is chosen for implementation. But for any app that can be moved, it can be put into containers and moved far more easily in the future. For many apps, it is simply dropping it into the new environment and configuring networking to route traffic to the new location.

    We’re already going to leave future teams with a hodge-podge of databases, programming languages and networking tools such as WAF/load balancing. Let’s make life a little easier on them, and help increase the use of DevOps, by deploying with containers and targeting whatever makes sense for the org today, secure that tomorrow that can change and our burden will be less than it might have been.

    — Don Macvittie

  • IT Enables Scale in Digital Transformation

    IT Enables Scale in Digital Transformation

    Few corporate functions have seen their role disrupted as much as IT these last few years. The help desk days are long gone. As digital transformation accelerates, IT is becoming less about outright technology and more about creating capabilities that enable the overarching business strategy. As such, IT is the engine behind any company’s digital transformation.

    Here are three ways IT can advance today’s digital journey by empowering the business from a consultative, strategic position.

    Going Digital While Digitizing

    First, let’s agree on digital transformation’s foundational terms, which understandably elicit some ongoing confusion: “digitization” and “digital,” which the MIT Center for Information Systems Research has described as two discrete yet complementary things. In short, digitization is about leveraging technology to improve or enhance business processes, operations and offer development. By contrast, going digital is about giving customer value to digitization—with the goal of revolutionizing the customer experience and creating new digital business models. What’s the point of digitizing a process or product if the user experience is poor? 

    Balancing these complementary efforts is essential. It is the role of IT to ensure that digitization and digital can happen at the same time. 

    Rethinking the Economic Model

    To keep up, IT must move away from traditional demand-based models, enabling capabilities with a product mindset instead of a time-bound project mindset. The concept of capability (that is, a business area enabled by a consistent set of IT products) is quite critical to avoid creating one-off or point solutions that are just the means to answer to a specific demand. Instead, looking to IT as the enabler of corporate practices that are always evolving and improving (e.g., cybersecurity) is imperative. Embracing Agile methodologies will bring this approach to fruition.

    Pivoting to this kind of a capability-based model requires a modern API-based architecture, which fundamentally is the way out of building and/or replacing monolithic systems. As today’s world requires a fine balance of speed and frugality, a mastery of resource allocation, and a prioritization rigor, we must unburden resources aimed at replacing systems that are working just fine. 

    To that effect, Amazon Web Services has a great architecture principle referred to as “two is better than zero.” In other words, it is better to have two similar IT components emerge from fast-paced, parallel development than nothing due to project bottlenecks and unmastered interdependencies. Even if we want to drive re-use every time we can (and have mechanisms for that), we should not bog down overall transformation due to one weighty project. So thinking in terms of not just tolerating diversity, but enabling it, is a good way to built an architecture that can empower both agility and resiliency.

    Addressing Technical Debt Continuously

    Technical debt is perhaps every incumbent company’s biggest challenge when going digital. The cost of maintaining legacy and outdated systems is high, and heavy technical debt is another obstacle to achieving full digital transformation. Accenture consultants see this every day, describing this burden as a price that “hinders [companies’] ability to innovate and adapt in the digital age.” 

    This is where IT can come to the rescue by having a structured approach and engagement model designed to streamline systems and processes in order to free up resources for reallocation. First focus on eliminating legacy custom-built applications and localized tools based on criteria that are well known and shared (think of a scoring system). Here, again, resource allocation can play its part by ensuring the full cost of those legacies is being recognized by the organization. 

    A huge added advantage of a continuous effort to remove technical debt (beyond cost, the reduction of obsolete systems and a simplified IT landscape) obviously relates to security. Especially at a time when the average cost of a single data breach globally is $3.9 million. In fact, potential cybersecurity and IoT-related physical safety concerns “will pressure CIOs at G2000 companies to increase IoT security spending by up to 25%, temporarily neutralizing business productivity gains” by 2020 (IDC). Redirecting technical debt can offset part of this essential spending on security.

    Accenture notes that 79% of CEOs report that their organization is adopting new and emerging technologies faster than they can address related security issues. IT teams therefore must carry the security torch, determining and implementing ways to make security easier for employees while also protecting the end-to-end digital ecosystem. A continuous effort to reduce the technical debt is one way—among many others—to improve posture. 

    A Place at the Digital Transformation Table

    Indeed, these are exciting times for every business. Digital is all about creating customer value, guided by a better understanding of customer needs deeper in the enterprise. Since providing a great experience is not only about the user interface but also about the ability to mobilize the internal systems in order to deliver at scale, IT experts now have a major seat at the table for revolutionizing the customer experience at scale.

    The essential ingredients to full digital transformation are clear: digitizing and going digital, being able to embrace emerging technologies with an architecture that makes diversity efficient when required and creating cyber citizens that continuously reduce the corporation’s technical debt. Behind the scenes and now front and center, IT is pushing forward the transformation strategy for companies staking a competitive claim in today’s digital economy.

    — Hervé Coureil

  • Don’t Let Cultural Debt Bring Down Your Business

    Don’t Let Cultural Debt Bring Down Your Business

    Businesses today are faced with myriad challenges to innovate and optimize. In doing so, we often need to make decisions that borrow against the future. This can come in a tangible form, such as taking money from investors to fund growth, or in more inconspicuous forms, such as making tradeoffs in technical decisions. When we make such tradeoffs, we expect them to come at the cost of additional technical work in the future; however, these decisions can also borrow against an externality that is often overlooked: culture.

    Technical Debt

    Technical debt occurs in the development of software when the “easy path” is taken to solve a problem rather than using the best overall solution. The concept implies that the time saved now will need to be paid back later (with interest!) in the form of rework.

    Ideally, technical debt is incurred knowingly, i.e. a decision to take a shortcut is made deliberately to increase velocity, and it is flagged to be fixed before becoming a compound problem. In calling out technical debt as such, we can better analyze our decisions, considering their net present value and future opportunity costs. By documenting it, we gain better insight into the responsiveness and flexibility of the code base. Having a strategy around managing technical debt has therefore become an essential discipline in the management of software products.

    Technical decisions also have an impact on the culture of a company, though, and this facet of technical management is often overlooked. “Cultural debt” can be taken on unknowingly, and the unmanageability that flows from it can be difficult to ascertain.

    Cultural Debt

    Cultural debt is when we make a technical decision that borrows against the culture of the organization. Such decisions can introduce divisions between teams, deteriorate communication or even weaken the effectiveness of leadership. These types of decisions need to be made regularly throughout the software development life cycle, and it is therefore critical that we begin to identify them, call them out and manage them accordingly. If the interest goes unpaid we can end up with a toxic environment that is difficult and onerous to manage—we risk a paradox wherein attempts to innovate can stifle progress.

    A Closer Look

    Just as with technical debt, a business ideally will have a strategy for managing cultural debt rather than allowing it to accumulate and bog down the business. Typically, decisions that involve cultural debt will impact on areas such as:

    1. Mission: Ensuring connection to the company mission and that there is minimal variance in how the mission is understood by different teams and departments.
    2. Values: Effective and reliable transmission of company values, the ability to apply them consistently and ensure that they are clear and tangible.
    3. Communication: Fostering openness and the ability to discuss, providing channels for communication and using channels effectively with clarity of purpose and minimal “noise.”
    4. Visibility: Providing the ability for everyone to see up, down and across their organization.
    5. Attitude: Happiness of employees, their effectiveness and retention.
    6. Leadership: Ability for leaders to govern effectively, and influence across departments

    When making technical decisions, we need to think outside the box and consider how our decisions will affect these cultural dimensions of the business.

    To reinforce the concept, let’s look at some current issues that the modern enterprise is being confronted with:

    1. Domain-Specific Tooling – As roles become increasingly specialized, a best-of-breed approach provides the tools needed to perform complex tasks. Unfortunately, tools create silos, and more tools means more fragmentation. Furthermore, in a world where tools are proxies for business processes, domain-specific tooling can cut off teams from processes, reducing visibility and impeding communication. This mean the business will need to devote cycles to meetings, emails and other noisy distractions just to pay down the interest and get the visibility and coordination needed for cross-functional effective leadership. Paying down the debt requires tool integration, and without an infrastructure to reduce the cost of integrating, domain specific tools can be a heavy debt to carry.
    2. Customization – Often we see the introduction of new tooling without the requisite investment to customize them to the business. Tools such as ServiceNow or Salesforce are highly valuable for being specialized to their use cases, but they still require customization to be effective. For example, when customer data is stored in notes fields rather than using structures data captured in the tool, visibility and reporting is gradually reduced and employee frustration abounds. The decision to introduce tooling should be made with the understanding that customization will be required and must be maintained over time to keep pace with the organization’s culture as it changes.
    3. Bimodal IT – One strategy for enabling cutting edge technologies is the use of a “bimodal” structure with essentially two IT organizations that operate under different constraints. While bimodal IT may be valuable for encouraging adoption of new technologies, it also encourages confusion of responsibilities, cross-team rivalry, knowledge silos and double work. The longer the structure remains, the deeper cultural divide and the more difficult an eventual transformation will be. The decision to introduce new technologies or practices using a bimodal structure should be done deliberately, cautiously and with clear goals in mind.
    4. Technology Selection – Often we are presented with new technologies that promise to deliver great benefits. Let’s assume that the new technology passes a technical assessment and is found to deliver on the promises: e.g. a language such as Clojure or Kotlin really will simplify development and increase velocity, or a new delivery tool does indeed remove a key bottleneck. Adopting such a technology invariably creates some cultural debt—developers unfamiliar with the new language may feel alienated, new silos will form around knowledge and expertise and tribes could be forcibly broken apart. Technology decisions that ignore cultural debt will involve more cycles and greater risk, while those that recognize it will implement new technologies with those issues in mind, providing the cross-training and collaboration needed to ensure success.
    5. DevOps – The dirty secret about DevOps is that it involves the introduction of a number of technologies and technical practices that carry a large amount of cultural debt. This can actually tear an organization apart if that debt is not managed closely, and it is therefore why DevOps—despite being closely associated with technical practices such as automation, infrastructure-as-code and CI/CD—is regarded as primarily a cultural movement. In the enterprise, DevOps technologies threaten roles, responsibilities and livelihoods. The cultural debt actually does outweigh the benefits, and DevOps transformations need to focus on cultural issues first before attempting to implement the associated technologies. This is common sense that we learned from observing lean manufacturing transformations in the nineties, which involved a similar change situation where technical practices became a function of culture.

    Cultural debt, like technical debt, can be hard to detect. If your business does not have a strategy for paying off the debt, or at least one for paying down the interest, the result will be a heavy load of cultural issues weighing down decisions and processes.

    Key Measures for Managing Cultural Debt

    • Understand your stakeholders: In an agile world it is easy to overlook stakeholders. We can learn from the disciplines of business analysis and project management, though, where stakeholder analysis is a core componency and key practice in work management. It is not possible to do a thorough breakdown of stakeholders for every technical decision, but an actively maintained stakeholder matrix will increase your mindfulness and can be referred to for a free sanity check when needed.
    • Understand your culture: Your organization is filled with cultural artifacts—if you are not aware of them and taking the time to manage them, you will not be able to assess cultural impacts. These artifacts can include mission and value statements, organizational and tribal structures, meeting cadences and compositions, control systems, routines, rituals, symbols and even legends. Think through your cultural artifacts and ensure that they match intention and are understood by leadership.
    • Obtain buy-in: Getting buy-in means giving people a stake in the game. Imposing technology decisions on people will not only result in a requirements mismatch, but it fosters discontent and mistrust. You can make a large downpayment on your cultural debt by bringing affected parties into the decision-making processes early. This has been shown to be very effective in helping pass decisions even when they have negative consequences.
    • Embrace the ‘Three Ways’: We can borrow from the DevOps movement the Three Ways, which provide a solid foundation for undertaking any technical work: Systems thinking encourages considering how decisions impact the whole system, not just the local area; feedback loops allow continuous feedback and communication about impacts; and continual experimentation and learning ensures that the organization is adaptive and resilient. Leadership that embodies the Three Ways is issued debt on very good terms.

    Conclusion

    Peter Drucker is quoted as saying that “Culture eats strategy for breakfast,” and the examples we’ve seen here highlight this. The best technology in the world cannot compensate for a broken culture, and a broken culture can be the result of a succession of ill-considered technology decisions. Understanding cultural debt is a key differentiator in being able to leverage Conway’s Law rather than be trapped by it. A key aspect of managing technology going forward will therefore be to identify and control this cultural debt as it is being created, and ensure that the organization is able to function with full-capacity engagement.

    About the Author / John Rauser

    John Rauser is the IT Manager of Tasktop Technologies, a global enterprise software company. As part of his role, he is responsible for specifying, managing and administrating software lifecycle tools, witnessing first-hand the impact they have on users and the business on a broader scale. He also serves as VP Operations at the board of the Project Management Institute – Canadian West Coast Chapter, providing leadership and expertise on technology issues. He has deep passion for discussing the business impacts of technology and analyzing strategies for IT management.