devopsy.kinsta.cloud

Category: Platform Engineering

Platforming the Developer Experience
This article is part three of three on the boom in platform engineering and its role in supporting a streamlined, positive developer experience.

In a perfect world, the developer experience would be on that is completely free of obstacles. In the previous articles in this series, I argued that supporting and improving the developer experience is a critical business requirement. To continuously improve and keep up with the constant change that typifies the cloud-native development landscape, platform engineering is–at least for now–the golden path to reducing developer toil and increasing productivity.

I’ve also countered the controversial idea that DevOps is on life support because centralized developer platforms eliminate the need for DevOps. If anything, DevOps and developers share a symbiotic relationship, and the developer platform is where their functions converge. In this third and final article of the series, I will explore this relationship, examining the considerations for how the developer experience is “platformed” and by whom.

Optimizing Developer Experience: The Circular Economy of the Developer Platform

Let’s put it this way: The more relevant and useful the developer platform, the likelier the developer is to use it. If it reduces toil, cognitive load and makes shipping software easier, the developer has more time to focus on their core task: Developing features. With intuitive, self-service workflows and all the tools developers need, they rarely, if ever, have to think about ‘the how’ of getting their software into the hands of users. And this works if and when an organization does at least a couple of things right:
- The organization prioritizes the developer experience and empowers other parts of the organization to answer the question, How can we create the optimal developer experience?
- The organization puts resources behind understanding and building the best developer experience–and that’s where both the developer platform and DevOps teams as “fixers” ideas emerge.
Does this mean the “optimal experience” can’t be optimized? Does that mean developers cannot have input into their own (or more general) developer experience(s)? No. In fact, part of what makes the developer platform idea compelling is that developers don’t have to weigh in or make decisions on the platform or tooling. Still, it’s possible to let them have that freedom if the team or organization wants to. Bottom line: There is no one-size-fits-all developer platform any more than there is a single developer experience. This is why DevOps, along with PlatformOps, is an ideal complement to facilitating the “circular economy” of the developer platform. The platform is unlikely to stay static. And to keep it dynamic and responsive to the changing needs of the developer teams using it, the DevOps teams of the future can proactively remove, recycle, repurpose and restructure the way the platform runs and what tools are available in it, automating and evaluating … almost like a feedback loop on the platform itself.

Your Developer Platform is the Golden Path to Developer Productivity

The developer experience is built around the tools and workflows developers use and, in the case of a developer platform, the tools and processes you define and provide, i.e., a “golden path” to productivity.

Whatever that platform includes, it will demand that you answer questions about what tools, processes and best practices will be most effective for delivering the optimal developer experience. For example, what and how much should you build yourself? How should the platform be assembled to best serve the needs of your developer team? What is the most effective configuration? How much freedom can or should be given to your developers, depending on your business goals and the security, stability and scalability requirements of your organization and products?

Boiling down the route to the golden path, there are three critical points to help guide the creation of an effective internal developer platform:
- Treat your platform as a product. This is key to reducing cognitive load and paving the way to shipping software fast and safely. This also means that you are guaranteed to have an MVP and alpha, beta, etc. version of the product–and subsequent versions serve their purpose temporarily but are always works in progress.
- Realize you can’t have good DevX without good UX and develop accordingly.
- Focus on workflows and tooling interoperability; that is, a developer(-led) control plane. What is going to work for developers in practice to ensure that they can focus on key deliverables? How will you gather intelligence to discover when it’s time to make changes to the platform? Who will be responsible for this?
Paving and Using the Golden Path Developer Platform Simultaneously

Developer platforms can significantly improve developer productivity and the overall developer experience. But you can’t just build a platform and hope for the best without seeking developer input or understanding how developers work. Nor can you build a platform and decide it’s finished and forget about maintenance, enhancements and inevitable change. Developer platforms don’t exist as one-and-done items on a checklist and are themselves productivity tools that are in a constant state of flux.

Misunderstanding why you would platform the developer experience in the first place defeats the purpose.
February 21, 2023
Technical Debt is a Major Threat to Innovation

A survey published today suggested that technical debt is starting to take a heavier toll as IT environments continue to become more complex to manage.

The survey of 400 IT leaders at organizations with 1,500 employees was conducted by Foundry, the market research arm of International Data Group (IDG) on behalf of Insight Enterprises, a provider of IT services. The survey found that 86% of respondents reported their organization was impacted by technical debt over the past 12 months, with the top areas affected being the ability to innovate (43%), meet service level agreements (41%) and avoid downtime (37%).

Technical debt is created when organizations cut corners and leave known bugs in the race to meet application delivery deadlines. Application code is often not as thoroughly tested as it should be. In some cases, it turns out that what was intended to be a minimum viable product doesn’t scale all that well. Quick fixes needed to improve performance are also frequently undocumented. Inefficient code also winds up consuming more IT infrastructure than required, which adversely impacts everything from cloud costs to the amount of carbon generated.

Juan Orlandini, chief architect for the Cloud + Data Center Transformation (CDCT) division of Insight Enterprises, said every organization struggles with technical debt; it will never be eliminated, but it does need to be contained, he added.

The challenge is instead of trying to keep the proverbial IT lights on during the COVID-19 era, IT teams are now being asked to innovate to advance digital business transformation initiatives, said Orlandini. A full 87% of survey respondents cited modernizing critical applications as a key success driver.

As a result, many organizations are embracing platform engineering to bring more structure to their DevOps processes, he noted. The challenge, however, is striking a balance between a more centralized approach to DevOps and maintaining the ability of developers to innovate, said Orlandini.

The issue, of course, is that in addition to massive technical debt, the latest generation of applications are more distributed than ever. The survey found 91% of respondents now rely on multiple public cloud providers for different workloads, with 54% of data residing on a public cloud. However, the survey also found on-premises IT environments are still relevant, with 20% planning to repatriate select public cloud workloads to an on-premises model over the next 12 months.

Nearly three-quarters (72%) of respondents also noted data volumes are growing faster than their ability to manage them, with 64% ranking implementation of a data governance program to improve data management as their top data-related objective this year. Just over a third (35%) of the data estate is now, on average, driven by edge workloads, with 29% of respondents reporting that at least 50% of data volumes now reside on the edge. Nearly two-thirds (64%) expected that to increase in the next 12 months.

A total of 85% of respondents also indicated their organization is using artificial intelligence (AI) to some degree to drive business insights and/or efficiencies, although only 36% reported having optimized their use of AI and machine learning (ML) across the enterprise.

Finally, cybersecurity also remains problematic, with 51% reporting they were impacted by a cybersecurity breach over the past 12 months. Just 20% resolved their issue in one day, while 31% needed less than a week. Nearly half (49%) required one week or more to recover.

It’s clear that IT isn’t becoming any easier to manage. IT teams need to find ways to build and deploy new applications while reducing technical debt so it doesn’t strangle future innovation.

February 16, 2023
DevOps Done Right: How to Succeed in DevOps From Day One
Whether you are diving into DevOps for the first time or trying to do it right this time, DevOps Onramp provides all the information you need for a successful DevOps journey. Join us on May 4, 2023, to learn how to get started with DevOps—and how to do it right.

You’ll learn:
- DevOps best practices, processes and the tools you’ll need
- Tips and tricks from experienced practitioners sharing their real-world experiences
- The basics of DevOps
- How to choose the right technologies
- Common challenges and how to overcome them
- The benefits of DevOps as a methodology and a career
- And more!
As organizations face increasing pressure to adapt faster and become more agile, getting DevOps right and adopting a DevOps mindset is crucial to deliver high-quality software faster and increase business value. And that opens up great opportunities for anyone looking to start a new career or pivot from an existing role in technology.

DevOps has become the standard of software development methodologies and it’s critical for successful digital transformation. Adopting DevOps—and doing it the right way—can benefit not only your organization but also your customers and end users by fostering innovation, reducing time to market, increasing development productivity, improving product quality and reliability and reducing costs.

That’s why many big companies, such as Amazon, Netflix, Facebook and many others have adopted a DevOps mindset—and DevOps practices. And as more businesses are looking to move to the cloud and transition into DevOps, there are plenty of job opportunities available for DevOps engineers at various levels.

DevOps is a growing field with a lot of opportunities for professional growth. According to a research study by Global Market Insights Inc., the DevOps market is expected to surpass $30 billion by 2028. Due to the high demand for skilled DevOps engineers and the shortage of qualified DevOps professionals in the industry, there is no better time to upskill than now.

Whether you are diving into DevOps for the first time or trying to do it right this time, DevOps Onramp provides all the information you need for a successful DevOps journey.

At DevOps Onramp Virtual Summit, taking place on May 4, 2023, you’ll get all the knowledge you need to get started with DevOps —and how to do it right. Join us to learn about DevOps best practices, processes and tools and get tips and tricks from experienced practitioners sharing their real-world experiences. We’ll go through the basics of DevOps, how to choose the right technologies, common challenges and the benefits of DevOps.

One thing is for sure: The future of DevOps is very promising. The rush has started—don’t be left behind!

If you’re interested in becoming a sponsor, please contact us. For more information, please visit the DevOps Onramp website.
February 14, 2023
The Evolution of DevOps: Developer Productivity and Platforms

This article is part two in a series of three on the rise of platform engineering and its role in improving the developer experience. Read part one here.

Developer experience is business-critical in organizations of all sizes and across all industries. In the first article of this series, I make the case that the cloud-native developer experience will be shaped and eased not only by platform engineering but also by the vital support of DevOps and/or PlatformOps teams. This flies in the face of the recent and somewhat controversial assertion that DevOps is facing extinction. In fact, it’s possible that DevOps is more important than ever, even if the role of DevOps has changed in the era of cloud-native development, supported by centralized developer platforms.

Business stakeholders who depend on reliable, stable, scalable and secure software shipping fast and consistently understand the business case for an integrated DevOps team, and most developers and their teams understand the challenges of coding, shipping and running cloud-native software without the guideposts offered by a platform (and the DevOps team). The complexities of Kubernetes and microservices (and their contingencies) make getting to productivity difficult, which is why developer platforms are touted as the magic-bullet solution. But these platforms don’t build and maintain themselves, and the concept hasn’t quite reached the level of maturity at which developer platforms are available as commercial, out-of-the-box offerings.

Developer Platforms: Who Paves the Developer Path?

DevOps is not dead and claiming that it is ignores the need for the surrounding, supporting roles that pave the developer path. DevOps is evolving, in some cases into PlatformOps, and keeps workflows flowing and manages operations that support developers’ ability to ship their work into the world.

Who paves the developer path? It’s actually a cooperative process between developers and DevOps/site reliability engineering (SRE) teams. In discussions with DevOps and SRE leaders about cloud-native development, one trend was a throughline: DevOps and SREs continue to play an instructive role in helping guide developers to realize the possibilities of self-service through supporting platforms.

The level of full-service ownership this amounts to for developers varies depending on the organization and its goals. But the role of DevOps is still clear. For developers, there is a significant learning curve, and DevOps can help clear the way to the centralized developer platform. This platform, assuming all goes as planned, will not only improve the developer experience but also relieve DevOps teams of the firefighting that has been typical of their job until now. Paving the path provides an opportunity for DevOps/SREs to both develop better operational support mechanisms for developers as well as engage in more long-term and strategic operational work.

DevOps Ways: Unlocking Developer Productivity via Developer Platforms

Helping unlock developer productivity is one aspect of powering a positive developer experience. And the shift to a developer-first platform model reinforces the business commitment to the developer experience. It becomes defined by its optimized tools and workflows, which make up developer platforms. The platform approach also supports wider adoption of cloud-native development across larger organizations and broader industries beyond the more niche applications we’ve seen, thanks to clearer guideposts and abstractions.

But becoming dev-first doesn’t kill off ops, which has been one recent point of contention. In fact, the three ways of DevOps continue to be critical:

● The First Way: Flow/Systems Thinking
● The Second Way: Amplify Feedback Loops
● The Third Way: Culture of Continual Experimentation and Learning

Quoting industry luminary Gene Kim, “The First Way emphasizes the performance of the entire system, as opposed to the performance of a specific silo of work or department—this as can be as large a division (e.g., Development or IT Operations) or as small as an individual contributor (e.g., a developer, system administrator).” Essentially, it doesn’t matter who constructs the platform as long as they are thinking about the entire value stream, starting with customer requirements which, for an internal platform, are driven by developers.

The Second Way is focused on creating right-to-left feedback loops. To quote Kim again, “The outcomes of the Second Way include understanding and responding to all customers, internal and external, shortening and amplifying all feedback loops, and embedding knowledge where we need it.” For platforms, this hints at embedding both developers in platform teams and ops in development teams.

The Third Way is focused on creating a culture that fosters two things: Continuous experimentation, taking risks and learning from failure and understanding that repetition and practice is the prerequisite to mastery. The platform you build today won’t be the platform you need tomorrow. This is why the ‘big bang’ approach to building platforms doesn’t work. Think big, start small and iterate fast. This is something DevOps is designed to do.

In supporting the cloud-native paradigm, DevOps is far from dead. It’s a shifting discipline that involves a focus on centralizing workflows and infrastructure to improve the developer experience and reduce their cognitive toil. This toil stems from developer guesswork, and DevOps is best positioned to remove that obstacle. If we concede that developer experience is central to business success, and devex depends on being able to code, ship and run quickly and safely, DevOps is the fixer who enables crucial self-service access for developers and, as we will explore in the third and final part of the series, “platforms” the developer experience.

February 14, 2023
flox Unveils More Accessible Instance of Nix Framework

This week, flox launched a more opinionated implementation of the open source Nix framework that programmatically automates package management and system configuration.

Fresh from raising an additional $16.5 million in funding, flox CEO Ron Efroni said the company will also make an enterprise edition of its namesake flox framework available later this year.

The Nix framework has been available since 2003 and provides DevOps teams with a portable, fully isolated development, test and production environment along with a catalog and tools for managing software packages. There are more than 5,000 contributors to the platform, but Nix is based on a unique programming language that many DevOps teams have found challenging to master.

The flox framework provides a higher layer of abstraction for invoking Nix, and provides a set of guardrails to make the framework more accessible to a wider range of DevOps teams, said Efroni.

That approach will make it easier to share development environments and packages across teams, machines, individual developers and entire organizations. The framework also provides visibility into the software bill of materials (SBOM) for applications used in production environments to enable real-time vulnerability management and detection without scanning, noted Efroni. That capability makes it easier to discover dependencies spanning multiple architectures; many organizations today still try to track dependencies using spreadsheets and tools that don’t serve to make DevSecOps processes simpler to manage, he added.

It’s not clear how many organizations are using Nix today, but Efroni said that as software development management continues to become more complex across multiple platforms and architectures, there is a clear need for a different approach to enabling portability at scale. The biggest obstacle is overcoming the psychological biases many DevOps teams have developed toward Nix over the last two decades, he noted. A more opinionated framework should reduce the overall fear factor associated with Nix, noted Efroni.

Regardless of the frameworks employed, it’s become apparent that DevOps teams are struggling under the weight of all the code they must manage, much of it written in multiple programming languages. Many organizations that embraced DevOps early are now finding the platforms they deployed have become challenging to manage and maintain. As application environments become more complex, there is a need to modernize DevOps workflows. Less clear is whether those efforts might be focused on, for example, containers and serverless computing frameworks versus a framework such as flox that is designed to span multiple architectures and platforms.

It might be a while before IT organizations develop a unified approach to application development and deployment, given all the preferences of individual development teams. However, with the rise of platform engineering, it’s apparent more organizations are trying to strike a balance between developer freedom and the need to reduce the total cost of DevOps. That cost continues to rise as additional software architectures and platforms are employed.

In the meantime, flox plans to contribute to both Nix and its own more opinionated version of the framework in the hopes of one day making it easier to achieve that goal.

February 9, 2023
Platform Engineering: Creating a Paved Path to Reduce Developer Toil

This article is part one of a series of three on the boom in platform engineering and its role in reducing cognitive load and creating a better developer experience.

Developer toil and cognitive load in the cloud-native space is real. The complexity posed by microservices, Kubernetes, and “software-defined everything” almost necessitated that ops needed to solve many of these issues to support cloud-native development.

Developers taking on full dev life cycle ownership and the requirement to shift left have resulted in cross-functional and operational concerns that have added even more complexity to already challenging workloads and unclear workflows.

To fuel productivity and consistency and enable developers to build and ship apps faster and more securely, one must understand what a ‘paved path’ might look like and where responsibility for different aspects of the development life cycle resides.

Supporting the Majority of Developers Without Alienating the Rest

Most developers want and need paved paths, particularly as paradigms shift; the adoption of cloud-native development being one such shift. These paths must help them reach the “safety” and productivity on which they are measured in their daily output and provide focus for completing their tasks. Freedom to make choices about processes and tools is not a priority because it ends up being noise that adds to the cognitive load and, in most cases, does not serve the company’s software shipping goals. This majority needs tighter guardrails and guidelines to do their work and get their code into the world without having to figure out too many operational parameters for themselves.

The cloud-native tooling landscape is vast, duplicative and overwhelming. Just trying to select and test tools could be a developer’s full-time job. Likewise, many organizations and developers have reported frustration—they are wasting a lot of time on repetitive work and routine tasks that deliver no business value, such as setting up environments and troubleshooting CI pipelines along with slow feedback loops in the development process.

As organizations increase pressure to deliver more with less and do so more frequently, these responsibilities make for a poor and unproductive developer experience and a far-from-ideal use of organizational resources. Stack Overflow’s 2022 Developer Survey reported that 62% of respondents spend more than 30 minutes a day fixing problems and looking for answers, leading to somewhere between 333-651 hours of lost time per week across a team of 50 developers. This reinforces a need for upfront investment in the paved-path developer platform, which eliminates much of this frustration.

It’s worth noting there is also a so-called 1% developer–one who wants to experiment and discover new tools on their own. Defining the developer experience within narrow guardrails ends up stifling this minority, which not only hampers their productivity but could stand in the way of innovation for organizations as a whole.

Building quality software requires accommodating both kinds of developers. For most development teams, the goal is to give them a clear route to building apps faster and shipping safely by prescriptively paving the path. The path also must provide enough freedom for adventure, allowing developers to stray from the path (within reason and where it makes sense).

Internal Developer Platforms: Paving Just Enough Path With Platform Engineering

Platform engineering and internal developer platforms (IDPs) are much-touted approaches to taming the toil. It is a part of the solution, and as it gains traction, it is becoming more mature. That is, instead of developers and development teams taking on full responsibility for every phase of the software life cycle and potentially cobbling together something resembling a developer platform, the industry trend is moving toward more comprehensive, commercial out-of-the-box platforms. Whether or not this addresses the actual needs of the development organization, this shift works because developer experience is now a business-critical requirement, and a centralized developer platform facilitates that experience (with flexibility enough to house all types of developer experience and developers working across industries).

The developer platform concept, supported by architecture, DevOps and SRE teams, contains the ingredients of transformation both for business and for developer experience. As the concept reaches critical mass and the “productifying” of developer platforms matures, one thing seems likely: By shifting to a platform model, development organizations are moving from ops-first to developer-first Kubernetes. The developer experience becomes defined by its optimized tools and workflows (and enables wider adoption of cloud-native development across larger organizations and broader industries).

Refocusing Developer Efforts: Abstractions, not Distractions

Cognitive toil will exist for developers regardless of the tools they’re offered. Technologies grow more complex and change frequently, and there are performance metrics that must be met. Taking the focus away from developers’ primary concerns—coding and shipping software and features—contributes to this toil without generating much return. As much as developers should be able to gain enough understanding of the way their code behaves and the consequences it can have downstream when shipping and running it, they should not need to become DevOps experts as a part of their work. This is why DevOps, despite predictions to the contrary, isn’t dead.

The paved-path developer platform gives developers exactly that: The ability to shift left without drifting right. The platform provides much-needed abstractions and tools that enable developers to refocus on priority work without distracting or overwhelming them with too many choices.

Becoming dev-first doesn’t kill off ops, either. Dev-first Kubernetes and a centralized developer platform elevate the importance of the developer experience while continuing to highlight the importance of DevOps in supporting and optimizing the developer experience. This is an entire topic in and of itself, which will be explored in part two of this article series.

February 7, 2023
Five Great DevOps Job Opportunities

devopsy.kinsta.cloud is now providing a weekly DevOps jobs report through which opportunities for DevOps professionals will be highlighted as part of an effort to better serve our audience.

Our goal in these challenging economic times is to make it easier for DevOps professionals to advance their careers. Of course, the pool of available DevOps talent is still relatively constrained, so when one DevOps professional takes on a new role it tends to create an opportunity for others.

The five job postings shared this week are selected based on the company looking to hire, the vertical industry segment and, naturally, the pay scale being offered.

We’re also committed to providing additional insights into the state of the DevOps job market. In the meantime, for your consideration:

LinkedIn

Apple
San Diego, California
Site Reliability Engineer (SRE) – Video
$115,000 to $217,500

SimplyHired

ScientiaMobile
Reston, Virginia
Senior DevOps Engineer
$110,000 to $150,000

Monster.com

Larsen & Toubro Infotech Ltd.
Farmington, Connecticut
DevOps Engineer
$110,000 to $130,000

Indeed.com

TherapyNotes.com
Remote
Senior DevOps Engineer
$105,000 to $155,000

Dice.com

MRoads
Dallas, Texas
Platform Engineer – SRE
$100,000 to $120,000

February 6, 2023
Blameless Integrates Incident Management Platform With Opsgenie

Blameless this week announced it has integrated its namesake incident management solution with the Opsgenie alerting platform from Atlassian.

Aaron Lober, director of product marketing for Blameless, said the Blameless platform is now integrated with both Opsgenie and PagerDuty, the two most widely used platforms for managing alerts.

Blameless provides an incident management platform that normalizes data collected from a wide range of DevOps tools and platforms to determine how best to respond to different incident classes. Armed with those insights, it then becomes possible to quickly assemble the best team to respond to an event, said Lober.

That capability reduces stress because Blameless also surfaces recommendations for resolving incidents based on the data it has collected, he added.

The Blameless platform is tailored to organizations that have hired site reliability engineers (SREs) to manage IT operations using DevOps best practices, noted Lober. The overarching goal is to eliminate rote processes that, over time, conspire to burn out incident management teams that often experience high turnover rates, he added.

Opsgenie has been connected to the Blameless platform via the service catalog and alerting functions. Blameless users responding to or managing an incident can escalate issues to any teams defined within Opsgenie. The service catalog and alerting data generated throughout the life cycle of an incident is automatically captured and stored on the Blameless Incident Timeline. That capability makes it possible to identify appropriate service owners and the escalation protocol that was followed by that team. Blameless can also display the actual user name of each responder that has been notified.

When accessing Blameless via Slack or Microsoft Teams, Opsgenie can automate both when an incident channel is created and pre-select relevant services that should be made available. Alerts can also be manually triggered using commands from the Blameless chatbot for those who prefer to trigger alerts from within Slack or Teams themselves.

As IT environments become increasingly more complex, more organizations are making a concerted effort to apply DevOps best practices to incident management. The goal is to reduce downtime and increase overall resiliency at a time when organizations are more dependent on software than ever. As part of that effort, IT teams should identify the repetitive processes that span multiple incidents and then move to automate as many as possible.

In the longer term, it’s not clear what impact artificial intelligence (AI) will have on incident management. At some point, though, IT teams should expect their efforts to be augmented by machine learning algorithms capable of identifying issues before they balloon into a major crisis. However, there will always be some human intervention required to manage incidents for the foreseeable future, noted Lober.

In the meantime, most IT teams should continue to expect the unexpected at a time when IT environments have never been more dynamic and change multiple times a day.

February 3, 2023
How To Build Anti-Fragile Software Ecosystems

Just as organisms are susceptible to diseases and viruses, software systems are susceptible to hacks and errors. And within many complex interconnected systems, a minor bug could have a cascading effect across an entire organism, causing it to break. As such, increasing the resiliency of a software ecosystem is something engineers continually strive for. But how do we get there?

One concept to consider is anti-fragile. Being anti-fragile refers to the ability of a system to thrive in adversity and adapt to change. Specifically, with regard to software, anti-fragile can equate to responsive coding practices, such as auto-scaling or default backups, to mitigate unforeseen events. Chaos engineering, for example, is one method to expose edge cases and build with adaptability in mind. And with the introduction of intelligent automation, often fueled by machine learning (ML) and artificial intelligence (AI), it’s becoming more and more feasible to build anti-fragile software systems.

I recently met with Asim Razzaq, co-founder and CEO of Yotascale and former head of platform engineering at PayPal, to discover what it takes to build anti-fragile software. According to Razzaq, anti-fragile represents the next evolution of reactive, resilient software systems. And introducing anti-fragile concepts into your application networks can be done with just a few (relatively) simple steps. Below, we’ll define what anti-fragile is and see how this concept can be applied to software development and DevOps.

Understanding Anti-Fragile

As the adage goes, “What doesn’t kill you makes you stronger.” And it’s true—many natural systems tend to thrive under inhospitable conditions. As described by Nassim Nicholas Taleb in the book ‘Antifragile: Things That Gain from Disorder,’ anti-fragile systems thrive in adversity. For example, consider how human muscle grows under stress and load. Immune systems similarly adapt and respond to new viruses and improve the body’s resilience over time. So, how does this relate to software development?

Well, organizations need to develop highly-tolerant software systems to meet rising digital consumer expectations. Especially when under extreme loads, systems must be elastic to adapt and survive. To meet high user demands, as well as specific service-level objectives (SLOs) and agreements (SLAs), most software companies are already investing in cloud-native tools that improve reliability. For example, a hallmark of site reliability engineering (SRE) is introducing automation to enhance the stability of applications.

One example of gauging anti-fragile in software environments is chaos engineering, says Razzaq. Chaos engineering is the act of throwing a wrench into the machine. You randomly introduce bugs and malformed requests and attempt to create as much havoc as possible to see what happens. This could mean pulling out cables and restarting servers to gauge how software systems respond.

“Software that can respond the best adapts the best,” said Razzaq. For example, if you manually remove a few nodes from the cluster but the system knows it needs seven to function, it could automatically spin up additional nodes. Or if the whole cluster goes down, ideally, the system would automatically restart and scale to the necessary requirements. In essence, anti-fragile software systems can understand and adapt to new environments without much, or any, human input.

Tips to Construct Anti-Fragile Software Systems

So, how can we construct anti-fragile software systems — especially when real-world challenges are often unpredictable? Razzaq shared helpful advice to consider when crafting anti-fragile software architecture.

1. Start With Your Most Critical Systems First

First, introduce chaos engineering to your most absolute critical systems. Define these vital components at the heart of your product which must remain functional. For a payments processor SaaS, for example, this would be the infrastructure supporting its transactional engine, such as the login and payment microservices.

2. Determine How to Test Fragility

Once you’ve identified this critical infrastructure, figure out how to test for fragility in your environment. Introduce chaos engineering to the component to see how it behaves. For example, returning to our payment platform example, this may include switching off a payment transaction type to see how the system functions in an extreme scenario.

3. Decouple Components

If systems are too closely coupled, errors and outages can ripple across multiple software services. To avoid that, it’s a good practice to prevent tight coupling between components—especially between critical systems and non-critical ones.

4. Introduce Alternate Paths

One method to decouple is by introducing alternate paths. For example, if a credit card verification component goes offline, the system should naturally accept other forms of payment and automatically suggest another form, such as a bank transfer. If all systems are down, the system could still accept unverified payments in a queue and let this continue until the threshold reaches a certain risk level.

5. Learn From Past Events

The next frontier of designing anti-fragile architecture is developing self-learning, autonomous systems. Of course, there is a limit to how automation can respond to unforeseen issues. But by feeding a machine learning model with more and more experiences, it could deal with untested new scenarios more gracefully. Of course, AI has the potential to make harmful decisions and even introduce unintentional bias, so organizations should have accountability for the algorithms they oversee, cautioned Razzaq.

Benefits of Building With Anti-Fragile in Mind

The goal is to able to thrive and function even under duress. By identifying the elements of core infrastructure to keep alive and putting it through the wringer, you can discover frailties before they cause significant consequences. And by decoupling components and introducing alternate paths, software systems can be better equipped for whatever may come. Uplevelling system operations with increased automation and AI is likely the future of constructing more fault-tolerant applications.

“Software is everything in this day and age,” said Razzaq. “It increasingly plays a critical role in life-enhancing capabilities.” As such, the end benefits of developing an anti-fragile style could have hugely positive effects. In health care situations, it could make a difference in saving lives. For other less dire scenarios, increased fault tolerance leads to less lost revenue during peak traffic times and more consistent end-user experiences.

January 31, 2023
GitLab Strengthens Remote DevOps Management

GitLab this week moved to provide remote developers with access to a more secure continuous integration/continuous delivery (CI/CD) platform while at the same time adding a dashboard that makes it simpler to track the overall status of application development projects.

David DeSanto, chief product officer for GitLab, said GitLab Remote Development takes advantage of an instance of the GitLab CI/CD platform running on a Kubernetes cluster. That cluster is accessed via a Web-based integrated development environment (IDE) to eliminate the need for developers to create development environments on their local desktops.

That capability reduces developers’ time and effort on tasks other than code and gives organizations more centralized control over the intellectual property being created, noted DeSanto. Organizations will no longer have to worry, for example, about theft of a developer’s laptop that has sensitive data on it, he added. In addition, organizations will be able to more easily create “golden” images that can be consistently accessed by developers residing in multiple locations.

In addition to making it easier to centrally manage a distributed team of developers, GitLab is also adding a GitLab Value Streams Dashboard that provides senior executives with access to metrics that assess the overall health of an application development project. While GitLab has always provided metrics to DevOps teams, DeSanto said this dashboard, available as a private beta, makes it possible to identify, for example, bottlenecks that might result in delayed delivery of an application.

Value stream management (VSM) is, of course, not a new idea. The issue that DevOps teams are trying to come to terms with is whether they need to adopt a dedicated VSM platform. The other option is to simply provide a set of dashboards that enable senior IT and business executives to track projects’ progress without deciphering a set of much lower key performance indicators (KPIs). Previously, the GitLab platform surfaced a lot of nitty-gritty details that don’t provide the context senior executives required, noted DeSanto. The Value Streams Dashboard metric surfaces KPIs based on DevOps Readiness Assessments (DORA) metrics from Google alongside other flow metrics.

These latest offerings are part of a larger case being made for an integrated DevOps platform that reduces the total cost of application development by eliminating the need for organizations to integrate various tools themselves. Each DevOps team can employ as many of the modules GitLab makes available as they need. Once invoked, they take advantage of the same data store as every other element of the GitLab platform. That capability is becoming more critical as organizations embrace platform engineering to reduce the cost of application development using DevOps best practices, said DeSanto.

It’s not clear how much interest there is in replacing legacy CI/CD platforms that, in some instances, have been in place for nearly a decade. However, as the pace at which applications are being built and deployed continues to accelerate, there is no doubt that organizations of all sizes will need to modernize application development processes.

January 25, 2023