devopsy.kinsta.cloud

Tag: continuous innovation

  • A Pandemic Side Effect: Rampant Technical Debt

    A Pandemic Side Effect: Rampant Technical Debt

    On average, about a quarter of all IT budgets are allocated to handling technical debt each year. And this figure is set to rise, as 68% of organizations say their technical debt will increase in 2022. As digital acceleration continues to drive more IT investment across divisions, the technical burden almost seems like an inevitable outcome of delivering more software and amplifying end-user experiences.

    Unsurprisingly, IT budgets are set to increase this year. This has been a stable trend throughout the pandemic as even companies that are not traditionally technical increased their reliance on software. Most teams quickly moved to remote working arrangements and acquired additional software tools to keep internal operations afloat. Not to mention the new complexities and array of tools emerging to support the microservices design paradigm.

    Although new tech adoption is vital to enable key initiatives across a company, unbridled technical debt can also inhibit digital transformation initiatives. Software AG recently released its Situation Report 2022, a survey that analyzed the state of technical debt among IT professionals in 2021 and technology adoption forecasts for 2022. Below, I’ll review the key points from the report and consider how organizations can respond.

    Increasing Reliance on Tech

    The growth in this area is astounding—78% of companies say their technical debt rose in the last year. A full 38% of companies reported taking on a great deal more; at the same time, IT budgets are set to increase for most organizations in 2022. On average, this increase is expected to be 37.1% higher than 2021 spending. It’s easy to correlate increasing technical debt with these IT budget increases.

    A large portion of this increased spending is related to digital transformation. This checks out; modernizing existing technology is a top priority for 19% of tech leaders. Other top priorities include exploring new technologies (17%) and decreasing technical debt (15%). Though managing technical debt is not ranked as the highest priority for IT leaders, it still ranks as a pressing concern due to its ability to hurt innovation. The report found that 69% of companies admitted it has caused their digital transformation initiatives to slow.

    Perceptions of Technical Debt

    So, what are the leading causes of technical debt? It turns out that most stems from recent modernization and digital transformation efforts. In fact, 39% said new digital products were a major cause. This is followed by infrastructure modernization (34%), data analysis and integration (32%) and new customer-facing services (27%).

    Interestingly, not all technical debt is bad. Often, IT professionals say the benefits justify the expense. For example, 86% say that increasing product or service launches justifies taking on tech debt.

    Technical debt is also a result of the remote nature of today’s hybrid work environment. During the pandemic, businesses made many concessions to enable remote work and adopted new collaboration and development tools to enable productivity. As a result, 88% reported the pandemic increased their awareness of technical debt. The majority, 83%, also said the pandemic has increased acceptance of technical debt, a sign that many of these changes are here to stay in the new normal of working life.

    Responding to Technical Debt

    Not all investments lead to the same degree of technical debt; some platforms are bulkier than others. Or, specific software vendor lock-in schemes can prevent lift-and-shift which can swell a technology ecosystem. So, what areas are companies investing in and which are the leading causes of technical debt?

    In 2021, companies invested heavily in cloud computing (60%), integration (43%), IoT (39%) and 5G (39%). Investment strategies will change slightly in 2022, with 5G becoming the top most-invested area at 45%. Cloud computing drops to 42% and edge computing is rising to 35%. Mapping where new investments will occur can help organizations anticipate the repercussions. Of these investments, 60% of IT professionals ranked 5G as a leading cause of technical debt. Other high producers include cloud computing, IoT and blockchain.

    In 2021, companies spent 24.8% of their IT budget handling technical debt. Managing it presents some significant challenges—32% say siloed data and processes are a key challenge when addressing technical debt. Limited budgets, regulations and a lack of internal alignment also get in the way. Clearly, resolving the issue requires both inter-department collaboration and a hefty budget.

    Final Thoughts

    Technical debt is, arguably, unavoidable as companies adopt cutting-edge technologies to remain relevant. And, taking on more technical debt is often intentional; for example, new DevOps tools are helping increase deployment fluidity and manage complex cloud-native architectures. Or, new SRE platforms bring observability and monitoring capabilities to improve application reliability. A newer programming language may be more performant, while a low-code abstraction layer complements an engineer’s workflow. In all these scenarios, the benefits often outweigh the added debt.

    Yet, much of it is unintentional and is slowing innovation down. This could be shadow APIs or forgotten rotting legacy infrastructure. Sluggish code and un-optimized routing between applications can also be a drag on resources. The reason these situations sound murky is because technical debt, itself, is difficult to quantify. And, only those addressing it with adequate visibility into their stack will have a clear indication of how much debt there really is.

    Though the current pandemic has indeed amplified technical debt, things will always evolve and new trends will replace the old. In the digital era, some degree of tech debt is inevitable within a large organization. The key will be effectively managing it over time to maintain a healthy, lean codebase and progressively release features that meet the demands of new customer experiences.

    The Software AG Situation Report 2022 surveyed 700 IT professionals on their software usage and annual predictions at the end of 2021. For more insights, you can read the full report without an email gate or paywall here.

  • Developer Challenges in Commercial Banking

    Developer Challenges in Commercial Banking

    Much like many other business sectors, the banking industry has had to accelerate their digital transformation efforts as a result of the COVID-19 pandemic.

    Today’s retail banking industry is a far cry from where it was even just five years ago. Consumers no longer have to physically appear in their bank to process payments or open new accounts. Mobile and web apps feature intuitive experiences and sophisticated functionality. Overall, retail banks and their software development teams have done a tremendous job in creating tools that are integral parts of personal banking today.

    However, while significant gains have been made in retail banking over the last few years, the commercial banking sector is lagging behind. For all of the innovation within the retail banking experience space, the commercial banking space is still as fragmented and “untechnical” as ever. For developers, this means they have their work cut out for them.

    If you’re a developer who wants to delve into the banking space or if you’re just beginning to work on commercial banking products, here are a few things to keep in mind.

    Be Prepared to Work with Outdated Tech

    One of the biggest challenges developers will face is that, more often than not, commercial banks are relying on decades-old legacy infrastructure. 

    As banks have evolved, their IT teams, vendors and consultants have routinely built layers on top of and beside core banking systems. The problem is in many cases, these layers were designed and implemented when authentication delegation, microservices and event-driven architecture were either too novel or nonexistent.

    Therefore, developers need to resign themselves to meeting the banks where they are in terms of technology. For example, they might see files instead of APIs, obscure XML schemas (never mind JSON), unexpected throttling and batching in addition to an overall lack of more advanced tools such as webhooks, events, streams and reporting.

    Patience is Key

    While elsewhere in the IT industry speed is king, many large organizations such as commercial banks tend to operate at a much slower pace. This goes beyond just using a “slower” methodology or having a “slower” development process with lengthier iterations. Unlike in a startup, the development process within larger organizations is a part of a bigger business process — and business processes in banks take a lot more time and effort and are subject to much greater scrutiny. Developers should expect a slower pace and must understand the need to be patient. 

    Embrace the Unknown and Experiment (Responsibly)

    Banking, finance and accounting are very complex, intricate domains. Moreover, banks are large and complex organizations whose inner workings most developers are not generally familiar with. Therefore, developers may be afraid of making mistakes and opt for safety over creativity. This simply can’t be allowed to happen.  Of course, no one likes to make mistakes. However, mistakes are often a key building block in innovation. Instead of letting the unknown stunt innovation, developers should embrace this unfamiliar terrain as an opportunity to experiment. Granted, there need to be quality control steps in place to make sure that these experiments don’t cause havoc or send critical functionality off the rails. 

    Here are a few best practices that each role can follow to strike the right balance between experimentation and objective achievement:

    • Engineering managers:
      • Build a culture that encourages writing clean code followed by refactoring.
      • Ensure developers understand the business and prioritize addressing customers’ pain points.
      • Make sure the organization allocates sufficient capacity to accommodate the cost of innovation. 
    • Developers:
      • Don’t rush putting things into the core of your platform. 
      • Move side effects to the edges of your application and decouple them from the business logic. Moreover, strive for keeping business logic as pure as possible when it comes to functionality; for example, make sure the same inputs produce the same outputs without creating side effects or depending on them.
    • Everyone:
      • Use the Theory of Jobs-to-be-Done to better understand the customers’ needs. It takes much longer to understand how the banking industry works than to understand the customers’ jobs-to-be-done and their needs.

    Having these guideposts in place will help developers become more comfortable with sparking innovation in commercial banking and allowing creativity to flourish. Understanding and accepting these challenges will enable developers to build the sophisticated tools and experiences that have been long overdue in the commercial banking space.

  • The Disposable Development Environment

    The Disposable Development Environment

    Wait! Just wait a dadgummed minute! It takes days to build a development environment, and now I am supposed to read an article on a disposable development environment?

    Yes.

    It seems like it took an entire sprint to upgrade PHP! Now I am supposed to let them build something so they can trash it?

    Yes … yes, you are.

    Disposable DevOps Thinking

    DevOps is based on five pillars:

    • Culture
    • Automation
    • Lean Thinking
    • Measurement
    • Sharing

    Because we automate environments such that they are easy to share, we should be able to measure meaningful business impact. This model is a DevOps practice of modeling developer workflow for operational tasks.

    Why a Disposable Development Environment

    Log in to your workstation and delete the runtime for Java, PHP, etc. Remove, or change, a few config files and empty the trash. Now what? What if your entire team did this because they had a great idea? Panicking yet? What if getting started in any team was as easy as:

    • Cloning the project from source control
    • Starting the runtime environment
    • Obtaining caffeine
    • Getting to work

    The majority of developers today have a hand-built, artisanal, bespoke workstation setups with the following characteristics:

    • It takes days to onboard a new team member.
    • Innovation around runtime environments is stifled.
    • Moving from one team to another is the work of days.
    • Workflow is more like Ops instead of Dev.

    Let’s set some goals:

    • No more than 30 minutes for a new team member to make their first code commit.
    • No more than five minutes to throw away an environment and start over.
    • No more than five minutes for a developer to move from one team to another.
    • Runtime changes happen the same way as application changes.

    None of this is possible unless your teams can throw away a runtime environment at will, thus disposability is an “[-ility]” that we should expect to be delivered with our software.

    Disposable is Indispensable

    The most important concept is handling runtime of the application with a virtual machine (VM) on the workstation. The application code remains on the workstation. This means that developers still use all the tools and workflows they have today. If a mistake happens on the VM, kill it and start over. If it happens on the workstation, then its time to rebuild the laptop.

    At its core, the ideas borrows from common concepts in our production environment (and a specific tool example):

    • Hypervisor for supporting guest virtual machines on the workstation, such as Virtualbox
    • Provisioner to configure the guest machine, such as Chef
    • Source Control to track and manage changes to the application and the configuration, such as git
    • Glue so the provider and provisioner can be expressed as code and managed through source control, such as Vagrant

    Onboarding a New Team Member

    Have the new employee follow a set of directions. It should take no more than 30 minutes for the employee to install the tools and the application he or she is to work on. As a manager, this must be measured and tested relentlessly. New hires should be directed to keep notes on trouble spots and anything that takes a long time. They improve the documentation for the next new hire.

    Accelerate Innovation

    Developers say, “I wish I could upgrade to PHP X! I would be done in minutes, not days!”

    Developers are capable of such an upgrade, so the real problems are:

    • It takes a few hours to upgrade a language version.
    • If the attempt fails, it could be longer to unravel all the hand-rolled changes.
    • If the attempt succeeds, everyone on the team is committed to undertake a fraught journey to upgrade their bespoke workstation.

    With management asking about things such as feature velocity, time to value and internal hurdle rates, who is willing to take that sort of time only to succeed and cause everyone else on the team grief, too? This is why security upgrades are always done with such pain. They are delayed because they are “non-functional.”

    Now abstract the runtime to a VM and the configuration of the VM to code. Assume the developer wants to upgrade PHP.

    • Developer destroys the current VM with PHP 5.2.6 on it.
    • Developer goes into the Chef code (for example) and changes the PHP version to 5.3.29.
    • Developer launches a new VM with a single command.
    • Developer runs automated tests.

    The most likely outcome is some tests fail. With a bespoke environment, this is where the upgrade effort stops. Fixing all the tests is a commitment to getting the entire team to move in lock-step. On a team of just five people this is daunting. Developers are not project managers.

    With a disposable development environment, it takes five minutes to make the change. Developers know it is an identical effort for their colleagues, so the thinking changes.

    Given an arbitrary time window, the tests are fixed and the PHP upgrade saves the time or improves system performance or patches a security hole. To share this change across the team this same developer commits the change in the Chef code. They push it to the code repository and message the team, “Hey PHP 5.3.29 works, please adjust.” One person does all the work.

    Team members stop what they are doing. They stash the work they have inflight. They pull the changes from the code repository. They start the VM and Chef configures it. They unstash their work and get back to providing business value. In less than five minutes the entire team is done.

    Accelerate Security and Compliance

    What if a developer’s upgrade causes security issues? Changing something as important as the language version is meant to illustrate this power. But this is not the right question to ask. Instead, ask this one, “How do we ensure we are secure before we get to production?”

    Assume that any change in the development environment is reflected in production through the same process. For example, when code is deployed to quality assurance (QA), the Chef cookbooks run and they bring up the machine with the new PHP version. Why would there not be tests to show that the application is at least as secure as previously?

    Even more to the point: By making it easy to upgrade the base language, test the change and share it across the team, upgrades are much less risky.

    This thinking has some new assumptions:

    • You trust test automation at the application level. (common)
    • You trust the test automation at the infrastructure level. (uncommon)
    • You have automated security and compliance practices that check before a production release. (rare)

    As managers we must help our:

    • Ops team becomes better at the practices of code management since the infrastructure will be code.
    • Development team makes responsible changes in concert with operations, security and compliance groups.
    • Security teams adopt a security-as-code mindset to facilitate testing before promotion from development.
    • Compliance team joins the development effort so changes are already part of audit output. Compliance-as-code is already on the way.

    Business Outcomes of a Disposable Development Environment

    We hire new people when we need them. Taking two weeks just to get into a working state means we are delaying time to value. Delaying 30 minutes is less time than filling out the HR paperwork. We need this critical time to fill the need that caused the hire.

    Developers have more freedom in the infrastructure to try their ideas. The also have the safety net of putting it back like it was in less than five minutes when the idea does not pan out.

    Patches to the runtime environment for security can be applied at development time where they can be tested through the full deployment cycle. This takes minutes instead of days.

    If all teams are using a standard method and tool chain, then it is possible to adhere to the maxim, “Put your best people on your biggest opportunity.” Developers can move from Team A to Team B in less than five minutes and get started learning the code. They can move back in the same five minutes.

    Changes in the runtime happen at development time, and they are promoted and release with the application. So, perhaps most of all, if it works on the developers laptop then it will work in production.

    Without the ability to simply throw away the environment and easily start over, Dev and Ops cannot innovate toward these benefits. So go ahead and dispose of those bespoke workstations, and be sure to manage toward the benefits:

    • No more than 30 minutes for a new team member to make their first code commit.
    • No more than five minutes to throw away an environment and start over.
    • No more than five minutes for a developer to move from one team to another.
    • Runtime changes happen the same way as application changes.
  • Iterative Indicators: Measuring DevOps ROI

    Iterative Indicators: Measuring DevOps ROI

    It’s well-established that application of ROI measurements to IT investment remains an imperfect science, and the same goes for ongoing adoption of DevOps methodologies.

    The fuzzy math of vendor-driven ROI calculators aside, even the most respected industry watchers have long struggled to create concrete parameters for assessment of technology spending. This, in large part, has been related to the fact that the requirement for continued investment has outweighed the value of short-term metrics.

    At the same time, it’s hard to put a price on the influence that paradigms including mobility, virtualization and the cloud are having on today’s business world. The scope of the so-called Applications Economy is only truly beginning to be realized.

    For in-depth discussion of DevOps ROI, register today for the upcoming webcast “Maturity Metrics – Analyzing the 2016 State of DevOps Report” with noted industry expert Gene Kim.

    With DevOps, the challenge of ROI measurement evokes similar characteristics as with these other bellwether movements. The affect that leveraging DevOps to become more agile and respond adeptly to changing customer expectations will have, long-term, ultimately will be realized as much in organizational viability as in any immediate cost-savings or other factors.

    That said, the recently published 2016 State of DevOps Report, assembled by the researchers at DevOps Research and Assessment (DORA), goes further down the path toward creating meaningful ROI metrics than any other such previous effort.

    For starters, the 2016 report builds on previous year’s conclusions that so-called “high-performing organizations”—those already deeply engaged in DevOps practices—are “decisively outperforming their lower-performing peers in terms of throughput,” and this gap continues to widen.

    Moreover, DORA contends that high performers deploy applications code 200 times more frequently than slow performers and achieve 2,555 times faster lead times. Perhaps even more importantly, in particular as it relates to the ability to improve applications performance, the report estimates that leading DevOps practitioners lay claim to 24 times faster recovery times and three times lower change failure rates.

    The State of DevOps Survey, completed by roughly 4,600 technology practitioners, also found that high-performing teams spend far less time on so-called “unplanned work and rework”—most often required to address existing problems—allowing them to spend nearly 50 percent more of their time on new work. The latter brand of effort is typically aimed at adding or improving feature functionality—or, in simple terms, innovation.

    As increased software velocity and reduction of MTTR represent the most critical differentiators in today’s environment, where an end user may wait only seconds for an app to load before abandoning your brand altogether, these conclusions obviously carry impressive weight.

    Finally, based on a framework established by research analysts at IDC which suggests that hourly application downtime costs can range from $1.25 to $2.5 billion for a Fortune 1000 firm, and that the average cost of a critical application failure is $500,000 to $1 million per hour, the State of DevOps Report allows for the conclusion that high-performing organizations save an average of more than $91 million per year.

    Much of this advantage is found, the researchers note, in the form of lower applications change failure rates, or those updates to software code that result in subsequent performance issues or outages. While high-performing DevOps practitioners experience an average change failure rate of 7.5 percent, medium-performing organizations see that figure rise dramatically, to a whopping 38 percent.

    If the undeniable adage that “time is money” applies in today’s Applications Economy as much—if not more than ever—than these ROI calculations would appear to infer substantial meaning.

    Later this month, Gene Kim, one of DORA’s founders and a key contributor to the State of DevOps Report, will host a webcast with CA Technologies expert Aruna Ravichandran to discuss DevOps ROI and many other related topics. Register for that event today.

  • DevOps and continuous innovation accelerate the evolution of technology

    DevOps and continuous innovation accelerate the evolution of technology

    DevOps tools and principles have dramatically increased the pace of innovation and enabled the organizations that have embraced them and used them most effectively to achieve a major strategic advantage over competitors. The more the world runs and relies on software, the more DevOps can be the engine to automate and facilitate change.

    There was a time it took almost half a day to build a single car. Henry Ford managed to bring that down to two hours and 30 minutes thanks to the assembly line, but for over a hundred years most of the changes year to year in automobiles have required retooling manufacturing plants so updates and modifications are developed at a relative snail’s pace. Then Tesla came along with a car that is more or less just a framework for the software that runs on it and that can have new features and capabilities the next day through over-the-air software updates.

    John Jeremiah, technology evangelist and team lead for HP Software, was quoted in a post earlier this year saying, “It was just a few years ago when nobody had heard of Uber. Now, every single taxi company or taxi service is under pressure to react to a company that owns no taxis. Why is that? It’s because they’ve used software.”

    Tesla has an advantage over other automobiles because it can change the way its vehicles work or improve them with new features at the push of a button. Uber has an advantage over the traditional taxi industry because it has streamlined its service through a mobile app available on hundreds of millions of mobile devices and it can modify and improve the way it works with a simple update. Software has increased the ability for organizations to adapt more quickly and DevOps accelerates that evolution even faster through continuous innovation and the automation of routine tasks.

    There will still be things that require physical change—an updated vehicle frame in the case of Tesla or a new feature that relies on cutting edge technology only available in the latest smartphone models in the case of Uber. Software alone can’t do it all.

    The areas where software makes a difference, though, DevOps has a major impact. A recent study conducted by Gene Kim and a team of researchers found that organizations that have embraced DevOps tools and principles effectively have 8,000 times faster lead times and 30 times more frequent deployments than their peers. They also have 12 times faster mean time to recovery when issues arise and are twice as likely to exceed profitability, market share and productivity goals. Those are all significant statistics that impact the bottom line and separate the companies that win from their struggling rivals.

    Companies that can harness the power of DevOps and continuous innovation to develop new apps and capabilities faster and that have the capacity to push those updates out to existing customers have a significant advantage over competitors still slogging through “innovating” the old-fashioned way. It’s hard to compete with companies that are deploying 30 times faster than you and have twice the profitability and market share.