devopsy.kinsta.cloud

Author: Valerie Fenyn

Solving the DevOps Labor Deficiency

We all know there’s a chronic imbalance when it comes to DevOps talent: High demand and limited supply. Look no further than a 2021 DevOps Institute report which found that while 60% of organizations are hiring for such roles, 64% of IT leaders have a tough time finding skilled DevOps professionals. And it’s not just the typical pandemic staff shortages that you’ve heard about. The shift to remote practices has fast-tracked digital transformation across industries, spiking demand for developers.

So what exactly is the state of the DevOps labor market? Is it only tough for companies, or for candidates too–and can automation help? In honor of Labor Day, we asked a few IT leaders about this classic labor conundrum.

Labor Barriers on Both Sides

Elisa Hebert, vice president of engineering operations at Fairwinds, observed that DevOps hiring is difficult for both the hiring company and the candidate. “Teams have been chronically short-staffed for the last couple of years due to the COVID-19 pandemic, which resulted in companies wanting to hire senior DevOps professionals who can operate more autonomously,” she explained. “If you toss in a need for specialization–we work exclusively with Kubernetes, for example–finding that level of domain knowledge combined with senior-level experience is really tough.”

On the flip side, she said the industry practice of requiring years of experience for junior-level positions makes it difficult for inexperienced talent to get hired: “It’s an impossible wheel that needs to stop.”

Automation to the Rescue … Or Not

You might be thinking, “Well, why isn’t automation helping with these shortages?” It’s true that DevOps automation can lighten developer workloads. Efficient collaboration between development and operations teams can reduce friction; faster feedback loops can accelerate deployment. Leaders can reduce the number of staff needed for tedious tasks while their engineers can focus on more fulfilling work.

All of this can help retention efforts by preventing burnout and promoting a healthier workplace. As Balakrishnan Viswanathan, vice president of cloud operations at Demandbase, said, “DevOps straddles two important pillars: Increasing development velocity and increasing release quality. Automation is key because otherwise, burnout is very high.”

So it’s no surprise that DevOps Institute’s Upskilling 2021: Enterprise DevOps Skills Report respondents rated automation as the most important skill domain. They ranked proficiency in automating continuous integration, continuous delivery, continuous deployment, continuous operation and support and DevSecOps as the most valuable knowledge.

But here’s where the automation solution runs into the DevOps labor problem: The need for DevOps practitioners skilled in automation vastly outweighs the availability. “From a DevOps perspective, there are very few people across the tech space who have operated production systems at scale and enabled automation to reduce the toil, making these skillsets extremely valuable in today’s market,” said Viswanathan.

DevOps Recruiting, Hiring, Retaining: What’s the Solution?

Given these challenges, how can companies improve automation and relieve their overworked engineers? Hebert said IT leaders should “create scaffolding.”

“Fairwinds has an engineering leadership team that is cross-functional (dev, platform, SRE, R&D, services, UI/UX) and plans strategy and tactical execution together,” she said. “We’re greater than the sum of our parts, so while departures are a mix of sad and happy, it doesn’t break us.” She also advised companies to revise their job descriptions and “take all the extra stuff out that isn’t actually critical to the job,” pointing out that women typically don’t apply for jobs unless they meet every criteria listed.

Marcus Merrell, vice president of technology strategy at Sauce Labs, mentioned Historically Black Colleges and Universities (HBCUs) as a talent source, reminding leaders that “an excellent, under-utilized resource for recruiting is HBCUs and non-profit organizations that focus on CS education for girls and people of color. These organizations produce thousands of graduates every year who uniformly have trouble getting hired.”

Viswanathan recommended that organizations provide more automation guidance. “One thing companies can start doing immediately is establishing thought leadership in an effort to showcase all the automation work that their DevOps teams are doing by developing content, whether blogs, articles or white papers.”

Finding the Right DevOps Opportunity

There’s no silver bullet for a beleaguered DevOps job seeker, but Merrell encouraged candidates to apply for positions even when they don’t possess every qualification listed. “For people who identify as female and people of color, try to remember that the job requirements are usually only looking for about a 60% to 70% match,” he said. “Remember that the job description is aspirational and that no company really expects to match this 100%.”

Hebert also advised looking beyond salary. “If a company can offer you an opportunity to grow your skills by working on interesting projects or with senior-level engineering talent that can mentor you, that’s an invaluable asset.”

Defining DevOps Labor Solutions Differently

Some organizations may be coming up short on both automation and adequate staffing right now, but incremental improvements can help the industry get there. And IT leaders believe we will get there. “As cloud adoption gains pace, the skill sets and labor force needed are expected to grow in lock step,” predicted Viswanathan. Until then, both the DevOps hiring manager and the DevOps candidate can take steps to meet each other in the middle.

September 2, 2022
Technical Debt: When It Works and When It Doesn’t
Every software team has probably faced technical debt—the accumulation of little development deficiencies that will require rework down the line. Just like financial debt, they accrue interest as they build up, in the form of difficult software modifications or challenges in adding new features. Time is the usual culprit; your development team has one timeframe for creation but your leadership team wants to bring the software to market faster. And when it comes to choosing between building a clean and polished piece of software or hitting that delivery date, you usually need to bow to pressures from on high. It’s at that point that your collaboration might go off track, with little pieces of incomplete code jumping into the mix.

Without careful monitoring, technical debt will eventually slow down your development and deployment processes, degrade your product quality and limit your ability to innovate in a changing market. But is technical debt unilaterally bad? Not always. Sometimes it’s in service to a greater strategy. Let’s look at the differences.

When is Debt Part of a Good Strategy?

Pankaj Gupta, senior director, product marketing at Citrix, said when technical debt impedes an organization’s ability to function, that’s a clear sign they need to wipe out the debt and start with a clean slate.

“But there are a few scenarios under which it makes sense to let debt accumulate,” he said. “For example, if the cost of addressing a technical debt is significantly higher now than in the future; if the debt is not impacting your immediate and short-term business needs or if you have an emergency release like a major security vulnerability fix.”

Daniel Viveiros, chief technology officer at CI&T, recommended that teams keep the big picture in mind when making the right tradeoffs. “For me, it’s about putting the business first and understanding that constantly activating value is imperative for success,” he said. “Well-managed technical debt can be an effective tool to shorten the lead time. Deferring refactorings, modernization efforts and other common debts in order to prioritize deployments is a constant tradeoff the engineering teams deal with on a daily basis.”

What are the Signs of Too Much Technical Debt?

Gupta provided some warning signs for technical debt overload. “You know you’re accumulating too much technical debt when the cost and/or time to fix the technical debt is higher than starting fresh,” he said. “And the time taken for each release and deployment is consistently increasing.”

A less obvious result: a DevOps exodus. The development team will be the first to feel the impact of too much technical debt, such as having a harder time completing tasks and working long hours to fix frequent breakages. “Employees are leaving the company out of frustration from working on legacy systems, spending most of their time fixing ad-hoc issues, routinely missing and being on-call all the time,” Gupta said. What debt reaches that level, resignations begin to mount.

Viveiros pointed out that compiled debt will impact system resilience: “Bugs are originated more often and the change failure rate goes up. Speed is perceived as risky instead of something beneficial to the digital delivery process.”

The ultimate sign of too much debt, he said, is poor business performance and lost revenue. “If downtime equals lost opportunities, the technical debt will become a financial debt,” Viveiros said. “Too much technical debt may impact the perceived quality of the product. As a result, the product and brand’s image will suffer.”

What’s the Best Way to Manage Debt?

Failure to deal with technical debt means kicking it down the road to another developer to tackle—and no one wants to be that person. So what’s the best way to take control and pay it off? To start, approach it as a collaborative effort and implement these four practices.
1. Identify the types of debt. Not all technical debt is created equal. Distinguish between the debt you accept in the moment as something to fix later on and the inadvertent debt you discover—and categorize it into code debt, design debt, build debt and other forms to identify origin patterns.
2. Analyze and automate. Analyze the origins of your debt and look for ways to tighten workflows or automate certain tests and processes. By reducing common errors and hidden bugs, you can stop them from snowballing into technical debt.
3. Develop new policies and standards. These should clarify when debt makes sense and when it causes unacceptable damage. For instance, an immediate release of a security patch would be deemed acceptable while errors that will eventually cause considerable downtime would not.
4. Communicate the cost to IT and business leaders. When another hyper-speed deadline comes your way, make sure your leadership understands the risks to product quality and developer retention. With the price tag made clear, they’re more likely to change delivery dates or provide funding for additional developers.
Like most out-of-control spending, harmful technical debt begins with poor decisions and a certain level of denial. By approaching decisions with greater clarity and foresight on both the Dev and Ops sides of the house, teams can give themselves more breathing room and the space to produce exceptional software.
June 9, 2022
Three Common Testing Mistakes – And Solutions

A few weeks ago at SauceCon, devopsy.kinsta.cloud talked to Matt Wyman, chief customer officer at SauceLabs. This year’s theme, Reimagine Test, emphasized the need to prioritize the user experience in testing. As Wyman put it, “We really focused on that customer experience, using the right type of test tool at the right moment.” After the conference, we caught up with him to talk more about a critical point—that many teams don’t know how to test for user experience because that hasn’t been in their wheelhouse.

Wyman provided advice on three typical testing mistakes and how developers can improve the accuracy and breadth of their testing environment.

1. Bloated Testing Suites

Here’s a common scenario: Flaky tests get slipped into large test suites and developers ignore them instead of correcting or removing them. The suite gets increasingly bloated as people add more and more tests, which can obscure their results. “From the outside, it’s hard to tell which tests are the most critical when you’re making a change to pass,” Wyman said.

He explained that if two people are working on the same part of the codebase and person A sets up a focused series of tests, person B can make a change and be sure it will work when those tests pass. But that’s no longer the case with a testing suite stuffed with irrelevant tests.

“If instead, I create a two-hour suite with thousands of tests in it, you won’t know which one is most applicable to the change you’re making,” Wyman pointed out. “This is one of the more significant complications when people are less knowledgeable about building a good test suite or test practice.”

2. Using a Single Tool Like a Hammer

Wyman emphasized the need to use the right testing tools at the right time for the right purpose—something many developers don’t do. In this common mistake scenario, he said, the developer takes a functional user interface testing framework and uses it to validate their integrations, their APIs, their visual results and everything else.

“It’s a classic case where you have a hammer and everything looks like a nail,” he said. “It makes your overall testing practice very heavy and very long. Typically, if you’re not using the right tool for the right purpose, it takes a lot of effort to maintain it.”

3. User blind spots

You don’t know what you don’t know—or, as Wyman put it, “the experience error is hard to find and even harder to discover.”

Wyman discussed a hypothetical scenario in which engineers build and test a system that accepts only numeric inputs. Testing runs perfectly—because the engineers test only the entry of numbers. The fix here is fuzz testing: Thinking beyond simple use cases and introducing a “what if” quotient by testing random and unforeseen inputs, such as users who enter letters. Building this foresight into the programming language itself can involve developers in the quality assurance process much earlier and avoid mistakes down the line.

He also stressed the need for observability in testing, with instrumentation that can help teams predict where users might run into limitations. “Developers have to look at the user analytics and usage analytics, not just system analytics,” he said. “Quite often the user analytics and observability the product manager is looking at, they see from the outside—and that’s useful data for the developer, so they can see those failure cases early.”

Transcending Testing Limitations

Is there such a thing as a flawless test environment? Probably not. But by sidestepping the three mistakes above, developers can transcend ingrained testing paradigms and elevate both quality and experience.

May 31, 2022
Stytch Launches New, Flexibility-First SDK

Passwordless solutions have been a trend for a while now, improving user experience (UX) while reducing exposure to common attack vectors. Now, Stytch’s new JavaScript SDK aims to make password-free authentication a bit easier by increasing customization and flexibility while offering fully baked authentication out of the box.

Julianna Lamb, Stytch CTO, said the solution provides dual benefits by doing the heavy lifting of implementation while providing the flexibility to create a powerfully branded UX from the customer’s first sign-on experience. The Stytch JavaScript SDK can offer an all-in-one experience that handles the entire user auth journey with a pre-built frontend component. But it can also give teams complete control over designing a sophisticated and tailored authentication journey. In an era where tech companies chase the ultimate customer experience as their holy grail, this should appeal to teams wanting to improve a customer’s first impression of their brand, Lamb said.

Multiple Options for a Smarter Front Door

Like many authentication tools, Stytch’s product suite originally offered two options: One was a standard widget to handle frontend components while backend APIs provided logins. The other option provided direct API integration, in which the developers built their frontend, handled tasks like session management and setting cookies on the frontend and used the API to do things like send a one-time passcode or email a magic link.

“But we found people really want flexibility when it comes to any solution they’re using,” Lamb said. “They also don’t want to do the heavy lifting when it comes to things like session management. Those things can be complex to reason through and having those out of the box is really compelling.”

The new SDK decouples drop-in UIs from helper functions, giving teams multiple options. They can place Stytch’s UI components anywhere they want in the product, sculpting the authentication flow to their specifications. As a headless SDK, it can be embedded client-side in the application to handle all the calls to the API, do the authentication, create a session and check its validity. The result: Less backend code to write and less logic to implement.

Customizable UI flows mean that teams can fully own the UX experience and build a UI that is customized exactly how they want it to look—and they can do this without having to build an internal API to route authentication requests from client to server. Teams looking for ways to make a brand impact right out of the gate can capitalize on this ability by creating a unique user experience.

“Authentication is the front door to your application,” Lamb said. “Sign-up is the first impression the user has of your app—and the ability to customize it with your brand look and feel so the UI fits your aesthetic is super-critical.”

Frictionless Flexibility in Identity Verification

Because teams may have different authentication needs within one product, the SDK can be used in tandem with the core Stytch API to meet shifting needs. Teams could generate embedded magic links via the API while providing Google OAuth and Google One-Tap sign-in options via the SDK.

The solution also reduces friction by making it easier to not only authenticate users but verify them on return visits to continue the same session. By loading the SDK on the frontend, teams can automatically connect Stytch’s sessions product to the browser cookie, feeding user session data directly into the cookie.

The Stytch team is currently building out more SDK functionality and frontend components, Lamb said. Given the ongoing evolution of simple authentication tools that meet complex requirements, it will be interesting to see where innovative authentication goes next.

May 9, 2022
FusionAuth Adds Support for ARM-Based Architecture

Most developers need three things in life: Control, interesting projects and the ability to build on the technology of their choice. Now a new development promises to put all three in developers’ hands—FusionAuth’s announcement of its support for ARM-based architecture.

FusionAuth solutions offer multi-factor authentication, passwordless authentication and single sign-on, among other features. ARM-based architectures offer simplicity with a lower and cooler energy footprint. Translated into DevSecOps benefits, this means that FusionAuth just made it significantly easier for developers to meet identity and authorization requirements and strengthen their security when building on ARM architecture.

We talked to Daniel DeGroff, co-CTO at FusionAuth, to find out the details on how this support can benefit DevSecOps.

The Movement to ARM-based Architecture

You’ve probably noticed many forward-thinking developers are turning toward ARM-based technologies. Developers can count on interoperability across different implementations and different devices—and ARM processors offer a lower operating temperature and reduced power consumption. They’re also inexpensive to produce, making their price point attractive. This has earned them a place in everything from servers to smartphones to the world’s fastest supercomputer.

With many developers spotting the potential for higher performance and cost savings, it’s no surprise that vendors like FusionAuth would move to support ARM-based infrastructure, as well. By working with ARM processors, the company can support additional use cases such as IoT and high-performance cloud platforms that are built on ARM-based architectures. DeGroff described it as a pragmatic—and inevitable—move, and said, “If you want to use these new machines, you have to support this architecture in some fashion.”

“We develop on the Apple MacBook, and all new workstations will use Apple’s custom silicon which is based on the ARM architecture,” he pointed out. “ARM is generally faster and less expensive than an equivalent x86 server platform, and many of the low-power IoT devices will utilize the ARM architecture as well.”

DevSecOps Benefits: Greater Flexibility, Easier Security

FusionAuth’s ARM support can make the average security-minded dev’s life easier in several ways. FusionAuth runs on any platform, anywhere; by supporting ARM-based technologies, the company is extending developers’ ability to deploy its authentication and authorization platform how and where they want. “We have to ensure FusionAuth runs where our users want to develop,” DeGroff said.

In helping developers build on ARM-based architecture, FusionAuth also makes it easier for them to use an API to bring in FusionAuth at the early stages of app or software development. One of the platform’s selling points is that it makes it simpler for developers to solve typical authentication obstacles, like weak password policies or difficulty with privileged access rules, DeGroff said. By supporting ARM-based technologies, FusionAuth aims to help developers successfully meet even tough security requirements through flexible, sophisticated identity authentication and access tools—and do so earlier in their development cycles.

A Seamless Transition to Smarter Authentication

If you’re a developer interested in exploring the combination of FusionAuth and ARM technologies, DeGroff said any transition will feel seamless. “There is a great business case for moving workloads to this architecture,” he points out. Developers who want to take advantage of the improved performance and lower costs can begin to migrate workloads to ARM-based platforms now.”

April 11, 2022