devopsy.kinsta.cloud

Category: Blogs

Leading DevOps Transformations With AI

In my recent article Revolutionizing the Nine Pillars of DevOps with AI-Engineered Tools, I explained that AI-engineered tools can help with automating repetitive tasks, improving decision-making with predictive insights and facilitating proactive issue resolution. AI can drive a higher level of quality, security and performance in the software life cycle. Leadership, the first of the nine pillars of DevOps, is the catalyst for a successful DevOps transformation using AI-engineered tools. It’s the guiding force that brings about the alignment of people, processes and technology in the pursuit of continuous delivery and operational excellence.

How AI-Engineered Tools Assist DevOps Leaders

Vision and Direction: Leaders define the vision and strategic direction for a DevOps transformation. They set expectations, define goals and align all stakeholders around a common purpose. By clearly articulating the benefits and objectives of the transformation, leaders can secure buy-in from the whole organization and guide their teams toward success. AI can provide data-driven insights to assist in strategic planning and decision-making. By analyzing historical data and current trends, AI can help leaders make informed predictions about the future and set the direction accordingly.

Cultural Change: DevOps is not just about technology, it’s about changing organizational culture. Leaders are instrumental in driving this change. They model the collaborative, open and learning-focused culture that DevOps requires. They facilitate communication and collaboration across traditionally siloed teams and encourage a mindset of shared responsibility for product quality and customer satisfaction. AI tools can facilitate collaboration and knowledge sharing, helping to break down silos. These tools can enhance communication, support shared workspaces, and even analyze team dynamics and communication patterns to provide suggestions for improvement.

Resource Allocation: Effective leaders ensure teams have the necessary resources, including tools, training and time to successfully adopt DevOps practices. They advocate for their teams and prioritize investments in areas that will maximize the benefits of DevOps, such as automation tools or cloud services. AI can assist in optimizing resource management. Predictive analytics can help leaders identify where investments will yield the greatest return, and automation can free up team members’ time to focus on high-value tasks.

Risk Management: Leaders play a key role in managing the risks associated with a DevOps transformation. They set the pace of change, balancing the need for speed with the need to maintain stability and security. They make tough decisions about when and how to deploy new practices or tools and they are prepared to deal with setbacks or failures as opportunities for learning and improvement. AI can play a crucial role in identifying and mitigating risks. By predicting potential issues in the development and operations pipelines, leaders can take preventative action. AI can also improve incident management, speeding up the detection and resolution of issues.

Continuous Improvement: The nature of DevOps is iterative and based on continuous feedback and improvement. Leaders need to instill this principle within their teams, emphasizing the importance of regular reflection, learning, and adaptation. Leaders can drive ongoing improvement and innovation in their DevOps practices by fostering an environment where experimentation is encouraged and failure is seen as a learning opportunity. AI can support continuous improvement by analyzing performance data and identifying bottlenecks or areas for improvement. AI-powered analytics can turn raw data into actionable insights, helping leaders understand the impact of their actions and make informed decisions about future changes.

Risks of Not Leading With AI-Engineered Tools

In an increasingly digital world, the use of AI-engineered tools is becoming standard practice in DevOps. While every organization must assess its own readiness and capacity to adopt AI, the risks of not doing so are significant and could impact competitiveness and operational efficiency.

• Missed Efficiency Gains: AI offers substantial efficiency improvements through automation, prediction and analytics. Organizations not leveraging these benefits could find themselves outpaced by competitors that do.
• Decreased Competitiveness: AI can enable faster and more reliable software delivery, quicker response to market changes and more efficient problem-solving. Without AI, organizations might struggle to keep up with competitors in terms of innovation and speed to market.
• Inferior Decision Making: AI enhances decision-making by providing leaders with data-driven insights. Without these insights, leaders might make less informed decisions, potentially resulting in suboptimal outcomes.
• Reduced Agility: AI can help organizations anticipate and quickly respond to changes. Not using AI may limit an organization’s ability to adapt quickly to new market trends or customer demands.
• Increased Operational Costs: AI can significantly reduce operational costs by automating repetitive tasks, predicting system failures and optimizing resource usage. Without AI, organizations may face higher operational costs.

Transformation Roadmap With AI-Engineered DevOps

The successful integration of AI-engineered tools into DevOps is not a one-time project but a continuous journey of learning, experimenting and adapting. Leaders play a crucial role in guiding their organizations on this journey, and developing their own AI competencies is a key part of that.

1. Education and Awareness: Begin with enhancing your understanding of AI fundamentals, its capabilities, limitations and implications. This can be done through formal training, online courses, seminars, workshops or self-study.
2. Identify AI Opportunities: Leaders need to identify areas in their DevOps practices where AI can provide the most benefit. This might be where automation could save time, where predictive analytics could provide valuable insights or where machine learning could improve system performance.
3. Start Small and Experiment: Begin by piloting AI tools in one or two areas. This provides an opportunity to learn about AI’s capabilities and challenges in a controlled, low-risk environment.
4. Gather and Analyze Data: Data is crucial for AI. Leaders must ensure their organizations have systems in place to collect, clean and manage data. They also need to learn how to interpret the data and insights provided by AI tools.
5. Build a Multidisciplinary Team: AI requires various skills, including data science, software engineering and domain expertise. Leaders should aim to build diverse teams and foster a culture of collaboration between these different skill sets.
6. Scale and Integrate: Once they have seen success with initial pilot projects, leaders can start to scale their use of AI tools and integrate them more deeply into their DevOps practices.
7. Continuously Learn and Adapt: As with any technology, the AI landscape is constantly evolving. Leaders need to commit to continuous learning and stay updated with the latest trends, tools and best practices.
8. Develop Ethical and Responsible AI Practices: Leaders should ensure AI is used responsibly within their organization, considering aspects like data privacy, transparency and bias. They should create guidelines and practices for ethical AI use.

Pitfalls and Challenges

By being aware of potential pitfalls and planning for them in advance, leaders can increase the odds of a successful implementation of AI-engineered tools in their DevOps practices.

Resistance to Change: Implementing AI tools often requires significant changes to workflows and practices. This can be met with resistance from team members. Effective change management practices can help overcome this resistance. Communicate the benefits of the change, provide ample training and support and involve team members in the process.

Data Privacy and Security Concerns: AI tools often rely on access to sensitive data, which can raise privacy and security concerns. Develop and adhere to strict data management policies, ensuring compliance with all relevant privacy and security regulations. Use secure AI tools that offer robust data protection features.

Skills Gaps: Implementing AI tools often requires skills that existing team members may not have. Invest in training for your team, hire new staff with the necessary skills or bring in consultants or external experts to help.

High Expectations: AI is often perceived as a magic bullet that will solve all problems. This can lead to unrealistic expectations and eventual disappointment. Set realistic expectations about what AI can and cannot do. Communicate clearly that AI is a tool that can enhance efficiency and effectiveness, but it requires human oversight and cannot replace all human tasks.

Choosing the Wrong Tools: With so many AI tools available, it’s easy to choose ones that aren’t a good fit for your organization’s needs. Carefully evaluate potential tools based on your specific needs and constraints. Look at factors like compatibility with existing systems, ease of use, scalability and vendor support.

Lack of a Clear Strategy: Implementing AI tools without a clear strategy can lead to disjointed efforts and wasted resources. Develop a clear strategy and roadmap for your AI implementation. This should outline your goals, the steps to achieve them and how you will measure success.

Summary

Leadership plays a crucial role in the successful transformation and adoption of AI-engineered tools in DevOps practices. The journey involves educating themselves about AI, identifying suitable opportunities, experimenting with AI tools, gathering and analyzing data, building multidisciplinary teams, scaling AI usage and promoting ethical AI practices. However, challenges like resistance to change, data privacy and security concerns, skill gaps, high expectations and selecting the right tools can be encountered. These can be mitigated with effective change management, strict data policies, training and hiring strategies, setting realistic expectations, careful tool evaluation and a clear AI implementation strategy.

June 13, 2023
Survey Finds Cloud Management Strategies Shifting

Even though organizations continue to consume cloud resources inefficiently, they continue to increase the amount of money allocated. These were among the findings of a global survey of 1,000 technology practitioners and decision-makers conducted by Forrester Consulting on behalf of HashiCorp.

The survey found a full 94% of respondents reported that their organization struggled with cloud waste. Top factors contributing to avoidable cloud spending included overprovisioning resources (50%), idle or underused resources (50%) and a lack of needed skills (43%). Nevertheless, more than half of respondents (56%) said they boosted their cloud spending in the last year compared to 22% which reduced it.

At the same time, however, 92% of respondents said they are moving toward adopting, standardizing or scaling platform teams as part of an effort to centralize the management of cloud services, but only 39% are scaling the use of platform teams throughout the organization. The survey also found, however, that organizations are addressing skills shortages by adopting a common operating model/shared automated workflows (58%), reskilling/upskilling (56%) and working with partners (43%).

The survey also found many organizations are also committed to multi-cloud computing, with nearly three-quarters (73%) reporting this strategy is helping them reach their business goals. Another 19% said they expected this strategy to help them achieve their business goals in the next year.

The most-cited reasons for adopting a multi-cloud strategy were reliability (51%), cost reduction (48%), security and governance (45%), digital transformation (45%) and scalability (45%). Only 10% had no choice but to go multi-cloud due to acquisitions or other issues.

Security (88%), availability (84%) and scalability (84%) were identified most often as being important/very important multi-cloud success factors, followed closely by regulatory/compliance requirements (82%), staffing and skill levels (82%), budget (80%), visibility into cloud infrastructure (78%), automated tooling (77%) and platform teams (75%).

The top cloud initiatives were data protection (77%), followed by secrets management and access control (both at 75%), the survey finds. The top security concerns were password/credentials/secrets leakage (50%) followed by data theft (49%), phishing (46%) and ransomware (42%).

Overall, respondents said the biggest benefits of having a multi-cloud computing strategy were a stronger security posture (74%) and helping attract, motivate and retain skilled staffers (68%).

Chris Van Wesep, senior director of product marketing for HashiCorp, said it’s clear organizations are purposely using multiple clouds to reduce costs by pitting one cloud service provider against another. The challenge organizations face when using that approach is they need to define a consistent set of development processes that can be used across multiple clouds to keep total costs under control.

Organizations that embrace multi-cloud computing are also more likely to implement security best practices because of their need to consistently manage multiple cloud computing environments, he added.

Of course, there is always going to be a great disparity between high- and low-performing organizations. The survey found that about 25% of survey respondents are high performing, with an equal number falling at the other end of the spectrum. The rest fall somewhere in between those two ends of the spectrum. The issue is that the gulf between the high and low ends of that spectrum appears to be widening as more organizations fail to muster the level of skills and expertise needed to succeed in the cloud era.

June 12, 2023
Cycode Leverages eBPF to Secure CI/CD Pipelines

Cycode today added a Cimon extension to its application security platform that uses extended Berkeley Packet Filtering (eBPF) to thwart cyberattacks against continuous integration/continuous delivery (CI/CD) pipelines.

Alex Ilgayev, head of security research for Cycode, said Cimon uses eBPF to inspect network connections, running processes and file modifications within a pipeline to learn standard behaviors. Armed with those insights, Cimon will then surface any anomaly in real-time, he added.

DevOps teams can then craft and embed security and compliance rules within their pipelines based on the severity of the anomalies discovered, said Ilgayev.

eBPF makes it possible to run sandbox programs at the kernel level without adversely impacting performance. In effect, eBPF bridges the boundary between kernel and user space by enabling developers to combine and apply logic across multiple subsystems that, historically, were completely independent of one another. That approach enables, for example, a security tool to scale to the point where it can identify threats at much higher levels of throughput to improve overall scale at a time when the volume of cybersecurity attacks continues to increase.

Cycode is taking advantage of eBPF to enable DevOps teams to not only defend against known attacks involving malicious package installation, typosquatting, repojacking, dependency confusion, dependency hijacking and other types of dependency attacks but also zero-day threats, noted Ilgayev.

Cycode provides a platform for mapping metadata and events using graph technology that makes it simpler for application development and cybersecurity to consolidate alerts generated by their DevSevOps tools. Most recently, Cycode added a Cycode Application Security Orchestration and Correlation (ASOC) module that will automatically discover all the tools that make up a DevSecOps workflow without requiring any integration effort.

The approach enables the Cycode platform to discover, for example, dependency issues that would otherwise be missed by software composition analysis (SCA) or static application security testing tools (SAST).

While a lot of progress has been made in terms of adopting DevSecOps best practices, many organizations are still struggling with securing their software supply chains. Most of the members of a DevOps team have limited cybersecurity expertise, so they need the help of a cybersecurity professional to determine what specific actions are required to remediate a vulnerability. Cybersecurity professionals, conversely, don’t have a lot of application development expertise.

As the number of regulations specifically focused on application security steadily increase, every organization that builds software will soon need to embrace DevSecOps to better secure software supply chains. Unfortunately, developers have too often viewed cybersecurity as an obstacle to deployment while cybersecurity teams have historically considered developers to be a primary cause of the breaches they are expected to clean up. The first step toward bridging that divide is, of course, to realize that application developers and cybersecurity teams need to be invested in each other’s success.

June 12, 2023
Revolutionizing DevOps With Low-Code/No-Code Platforms

The DevOps industry is growing at an incredible pace, with its worth exceeding $8 billion in 2022. Analysts predict that the market will experience a five-fold increase and reach almost $40 billion by the end of the decade. But with this growth, there is a significant issue within the DevOps chain that continues to escalate: A widening talent gap. The U.S. Department of Labor predicted that by 2030, the global deficit of development engineers will exceed 85 million. Additionally, the demand for DevOps professionals is expected to increase by over 20% annually throughout the rest of the decade. These conflicting trends present significant challenges for software and application companies. While there is a substantial opportunity to generate revenue by creating new and better applications, the shortage of skilled personnel to develop these products is a significant roadblock. Integrating low-code and no-code tools into the DevOps process could be a viable solution to address the talent gap. These tools offer various benefits and efficiencies, including streamlining the work of existing DevOps professionals and enabling organizations to expand their teams beyond traditional personnel sources. Leveraging these tools could give companies in the DevOps industry a competitive advantage.

Challenges of Traditional DevOps

Traditional DevOps practices pose significant challenges for companies that hinder productivity and overall success. The first significant issue is the talent shortage. According to a report by the Manpower Group, 75% of employers consider finding tech talent their primary concern. As the demand for experienced DevOps professionals continues to grow, it is becoming increasingly difficult for companies to find the right personnel.

Another significant challenge in traditional DevOps practices is the high learning curve. The prediction by Gartner that 75% of DevOps initiatives would fail to meet expectations by 2023 highlights the severity of the learning curve and the challenges associated with adopting DevOps practices without proper training and experience.

To avoid delays, increased costs, and suboptimal productivity, companies should explore innovative solutions such as low-code and no-code platforms. This would enable citizen developers to contribute to the development process and streamline infrastructure management processes, enabling greater efficiency and scalability in their DevOps practices.

How Low-Code/No-Code Revolutionize DevOps

Low-code/no-code platforms are gaining widespread recognition for their potential to drive app development across organizations, as Gartner predicts that by 2024, 80% of apps will be developed by non-IT professionals. The adoption of these productivity tools can bring significant benefits to the DevOps landscape, particularly in the following ways.

Reducing Workloads and Stress
By using low-code and no-code software, organizations can distribute basic app-building and basic development tasks across the organization, freeing up skilled developers’ time and reducing their workload and stress. This significantly improves the workplace experience and enables developers to focus their time on delivering more complex, high-quality solutions.

Accelerating Innovation With Low-Code/No-Code Platforms
Low-code/no-code platforms can accelerate innovation by enabling developers to quickly spin up proofs-of-concept for more complex software development tasks that require coding expertise. Ideas and solutions can be tested before investing time and resources in full-scale development.

Creating New Roles and Opportunities
No-code platforms can create new roles and opportunities within the enterprise, such as no-code projects focused on optimizing processes and building solutions. Citizen developers without coding experience can pursue new career paths and opportunities to contribute to their companies’ success. One example is applying no-code technology to digitize paper forms across organizations. Employees without coding experience can create mobile apps that speed data collection, improve data accuracy and allow for instant digital data sharing and analysis.

Giving Developers a More Strategic Role
With tactical software development carried out within business units, developers can spend more of their time focused on cutting-edge technologies like mixed reality, quantum computing, and AI. This can give them a more strategic role in the enterprise and enable them to focus on the “next big thing.”

Considerations and Challenges

While low-code/no-code platforms have the potential to offer numerous benefits to DevOps teams, it’s important to be aware of challenges that have arisen when pairing the two successfully, and how to avoid them.

Poorly-Designed or Insecure Applications
With non-IT professionals developing applications, there is a higher risk of introducing vulnerabilities that could compromise the security of the application and the organization. Additionally, the lack of oversight and governance could lead to poor coding practices and technical debt. For instance, the use of new-generation iPaaS platforms by citizen integrators has made it difficult for security leaders to have full visibility into the organization’s valuable assets. Attackers are aware of this and have already taken advantage of improperly secured app-to-app connections in recent supply chain attacks, such as those experienced by Microsoft and GitHub.

Companies need to make sure IT staff carefully check how low-code and no-code technologies are adopted and how apps are reviewed in order to reduce vulnerabilities.

Limited Customization and Scalability
While low-code/no-code platforms can help speed up application development by up to ten times, their limitations can create hurdles when building large, complex applications. These platforms can be useful for basic application development, but they may not be suitable for complex or highly-customized applications. For example, if your app requires robust offline capability that handles conflict resolution and on-device offline storage, you’re better off selecting a low-code platform like Alpha Anywhere than a drag-and-drop app builder like Appypie, which is a fit for less complex apps.

Complex Integrations With Legacy Systems
As organizations try to integrate low-code and no-code applications with legacy systems or other third-party applications, technical challenges can arise. For example, if an organization wants to integrate a low-code application with an existing ERP system, it may face challenges in terms of data mapping and synchronization. Some low-code and no-code applications are built to export data and share it well, but when it comes to integrating event triggers, business logic, or workflows, these software solutions hit limits. Buyers need to consider their requirements carefully before making a purchase.

To address the potential challenges of low-code/no-code platforms, organizations must establish governance policies and procedures to ensure adherence to security and quality standards. This includes implementing proper access controls with zero-trust security, data privacy measures and vulnerability management processes. Organizations should work with their low-code/no-code platform providers to ensure seamless integration with other systems and scalability to support evolving needs. Along with this, regular assessments and audits of applications developed using low-code/no-code platforms can also help identify and address security risks before they turn into significant issues.

Conclusion

The integration of low-code/no-code platforms into DevOps practices represents a promising solution to address current challenges in the industry. By leveraging these platforms, organizations can expand their teams beyond traditional personnel sources and create a more inclusive environment for non-IT professionals to contribute to the development process. The rise of no-code is a win-win for everyone, and as early adopters seize this opportunity, they can gain a significant competitive advantage in the market. The future of DevOps is exciting, and the potential of low-code/no-code development is limitless, unlocking the innovation potential of all employees and empowering them to make a difference in their organizations.

June 12, 2023
Five Great DevOps Job Opportunities

DevOps.com is now providing a weekly DevOps jobs report through which opportunities for DevOps professionals will be highlighted to better serve our audience.

Our goal in these challenging economic times is to make it easier for DevOps professionals to advance their careers.

Of course, the pool of available DevOps talent is still relatively constrained, so when one DevOps professional takes on a new role it tends to create an opportunity for others.

The five job postings shared this week are selected based on the company looking to hire, the vertical industry segment and, naturally, the pay scale being offered.

We’re also committed to providing additional insights into the state of the DevOps job market. In the meantime, for your consideration:

LinkedIn

Underdog.io
New York, New York
DevOps Engineer
$114,000 to $206,000

SimplyHired.com

Verizon
Lone Tree, Colorado
DevOps Engineer
$107,000 to $226,000

Indeed.com

Erie Insurance
Erie, Pennsylvania
DevOps Solutions Architect
$104,838 to $167, 468

Dice.com

Nityo Infotech Corp
Mountain View, California
DevOps Engineer – AWS Cloud
$100,000 to $140,000

CareerBuilder.com

Boston College
Chestnut Hill, Massachusetts
Senior DevOps Systems Administrator
$96,000 to $120,000

June 12, 2023
A Life of Labor

June 12, 2023
Snyk to Add ASPM Platform via Enso Security Acquisition

Snyk this week announced it plans to acquire Enso Security, a provider of an application security posture management (ASPM) solution that tracks events and analyzes metadata collected from DevOps and security tools.

At the same time, Snyk revealed it has added a DeepCode AI Fix tool that creates validated fixes for code written by either humans or machines and an Insights tool that helps organizations better prioritize remediation efforts based on the risk to the business. DeepCode AI Fix extends the capabilities of a generative artificial intelligence (AI) capability based on large language models (LLMs) that Snyk gained with the acquisition of DeepCode in 2020.

Manoj Nair, chief product officer for Snyk, said all of these initiatives represent an effort to extend the reach of Snyk’s portfolio to address a wider range of issues within the context of a larger DevSecOps workflow as well as give developers tools that enable them to write more secure code.

The acquisition of Enso Security is expected to close this quarter. The company has been making a case for an ASPM platform that eliminates the need to manually collect cybersecurity data from the various tools and platforms that a DevOps team may be using.

Despite innumerable cybersecurity concerns, the rate at which applications are being deployed and updated continues to accelerate. The Enso platform makes it possible for cybersecurity teams to keep pace with those projects by discovering issues before an application is deployed in a production environment.

The challenge, of course, is that most development teams are not going to be able to address every issue before an application needs to be deployed. In addition to knowing who is developing what application code, cybersecurity teams need to also prioritize potential issues versus rather than just sharing a list of potential issues that have been discovered. Enso Security’s ASPM platform provides visibility into events so that it’s simpler for cybersecurity teams to trace issues back to individual developers or the AI platform that generated them.

That latter issue is becoming more significant as more development teams rely on general-purpose generative AI platforms to write code, noted Nair. Those platforms are writing that code based on samples collected across the internet, many of which are deeply flawed from a cybersecurity perspective, he added.

The Enso platform is intended to enable cybersecurity teams to have the most impact on an application development project without being forced to interrogate every developer to determine what’s occurring in any application environment. In fact, one of the reasons developers resent those conversations is they know all the relevant data has already been entered into a system—cybersecurity professionals just need to access it.

There’s still a lot of work to be done when it comes to bridging the historic divide between cybersecurity professionals and application developers. However, the more data that can be seamlessly shared between these teams, the simpler it becomes for organizations to embrace DevSecOps best practices.

June 9, 2023
The API Life Cycle Mixer’s Manual: What Goes Into a Single Source of Truth?

In bartending, the mixer’s manual is a trusted source of information. These books can contain hundreds of recipes for classic cocktails, some of which might be 100 years old, while others might be for drinks that only recently arrived on the scene. Mixer’s manuals also can be great sources of inspiration for new, yet-to-be-tested recipes for exciting additions to a bar’s or restaurant’s offerings. This balancing act of preserving the old while encouraging exploration into the new can provide a valuable lesson for API development in the software world. An API’s “single source of truth” is, in many ways, the software organization’s own mixer’s manual. This is especially true when the right approach is taken to ensure that software remains functional and performant and serves up experiences that delight customers and keep them coming back for more.

The Rise of APIs and Mixology

While we might think of mixology and the mixologists who practice this craft as relatively new terms, the first recorded use of the word “mixologist” dates back to 1852 and “mixology” followed shortly after.

Just how similar are the work, mindsets and passions of mixologists and API developers and designers? Check out this definition of a mixologist and count the similarities for yourself:

“The term ‘mixologist’ refers to someone who studies the history of mixed drinks, has a rich appreciation of the ingredients and techniques used and regularly creates new and innovative mixed drinks … their job title implies that they do a significant portion of their job behind the scenes, creating new craft cocktails and putting their signature twist on existing favorites.”

The best API developers, designers and architects of the world should recognize a lot of the above in their own work. They have deep respect for their craft and are always honing it further. They seek to understand the tastes and palates of their consumers. They factor in other stakeholders’ inputs, they test, iterate and test their creations again before “serving” those APIs to production.

Like mixologists and bartenders (if you see them as two different roles), every professional along the API life cycle should understand the needs of consumers, customers and the business–and then design and develop their APIs accordingly. An essential part of this process is continuous validation and adaptation to market requirements.

Maintaining Confidence in Ever-Changing Recipes

Changes to APIs, even minor ones, can have significant downstream effects. And the same can be said for tweaks made to an age-old cocktail recipe, where adding or removing a single dash of this or that can completely change the flavor of a drink … as well as people’s interest in it.

While there’s no “standard” rate of change to APIs to plan for, you can expect changes to be needed with each new software release. This could mean changes are being made (and need to be tested against and validated) every few weeks, every month or even every day, depending on your release cadence. There also can be as-needed, ad-hoc changes.

What’s crucial for teams to ensure is that these changes do not disrupt an API’s functionality or performance. To make this possible, teams are increasingly leveraging API contract testing to provide a safety net of sorts. This comes in the form of testing that validates changes against a “contract” that details an API’s original, agreed upon and, therefore, required functionality.

The ability to foresee the impacts of changes to APIs can depend on the maturity of an engineering team and its ability to collaborate across departments, time zones and varying levels of technical expertise. Tools and guidelines that promote standardization and collaboration can prevent mishaps from overlooking or being blind to negative downstream effects.

Documentation is Key

Whether we’re talking about being able to trust a single source of truth for the most up-to-date version of a published API or the latest edition of a mixer’s manual for tried-and-true drink recipes, clear and concise documentation is key. But it’s also not enough for this documentation to simply exist.

It seems like common sense—although you’ve probably worked somewhere where it wasn’t—that your API documentation also needs to be easily accessible, understandable and usable. Ensuring all of these things leads to smoother onboarding and a positive developer experience and, ultimately, to wider adoption of a functional, performant API that meets or even exceeds expectations.

API documentation is often the first point of contact a developer has with an API. It serves as a guide, walking developers through the API’s features, endpoints, request/response models and error messages. It also provides insight into the underlying business logic and unique characteristics of an API.

Good API documentation follows the principle of a single source of truth, meaning that it should be comprehensive, up-to-date, and consistent. It should always mirror the actual behavior of the API, reducing the guesswork for developers and minimizing the potential for misunderstandings and errors.

Just as a mixologist relies on a detailed and accurate cocktail recipe to deliver a pleasing experience to a patron, API developers depend on good documentation to create and deliver useful, value-add APIs and powerful applications.

Continuous Improvement = Continuous Education

Like a mixologist who meticulously adjusts the ingredients of a cocktail or who creates new cocktails to meet the evolving tastes of customers, developers must constantly adapt their APIs to meet changing market demands, business needs and industry standards.

This is achieved through a potent blend of market research, collaboration, adherence to standards and governance and meticulous testing. However, as software development, testing and deployment technologies and processes continue to evolve, the tech industry must continue educating itself and refining its practices.

With a mindset focused on constant learning and continuous improvement and a dependable single source of truth as their guide, development teams can confidently serve up exceptional API experiences in every release. In the end, the aim is to create APIs that are reliable and easy to consume and that meet the unique needs of the moment–much like a mixologist’s perfect beverage.

June 9, 2023
Technical Debt? No Sweat!

As a product manager, I dread the moment a developer says, “Wait a minute; I need to refactor …” (I wish it was just a minute). Something that should take two days starts taking three days and then a week. Over an application’s lifetime, the app starts accumulating technical debt. At that point, innovations become a smaller and smaller percentage of the work due to the vast efforts invested in the extra time required to support new developments.

It is okay and natural to get into technical debt. As in finance, some people will take out loans to invest in something because they know that later they will be able to sell it for a higher price and pay off the loan with ease. Same with code. You take on technical debt because you want to develop the application faster and get the product or updates out as soon as possible. But in the end, the technical debt has to get paid. This creates friction between R&D and product, and although we both understand at some point that modernization is needed, it is difficult for the team to quantify. Due to all the long cycles, the complex QA, the increasing number of bugs and security issues, engineers start to feel frustrated. And as a product manager, I also feel frustrated when dealing with missed deadlines and inflated estimations.

It’s not that we don’t want the developers to be happy and to invest in the code (and, let’s face it, in their own mental health, too), but we never know how much time and money is actually spent on technical debt versus how much is reallocated to innovation. For me, that was always a big question, but I never managed to move the conversation from, “I have a feeling,” or “The code looks ugly …” to “For every $1.00 I invest, X% is dedicated to maintenance.” How do we stop guessing and feeling we are in debt and actually start measuring it? And once we make technical debt tangible, when and where do we start modernizing?

Every application’s innovation cost is affected by its level of technical debt. It’s important to determine how much money invested in the application is put toward innovation and how much is used to repay technical debt. In addition, there are classes of debt that add more complexity and risk overall.

The newest versions of a monolithic application’s framework can be tagged as modern, aging or unknown (if the jar file isn’t found). This adds another layer of information when deciding what a modernization effort would entail and helps you focus first on aging frameworks that pose the greatest risk to enterprises due to the accumulation of technical debt and security concerns.

Complexity, Risk and Debt

The complexity of an application is determined by the degree to which class dependencies are entangled, reducing the level of modularity. A code risk relates to the length of the dependencies: How might a change in the application affect some of its seemingly unrelated components?

These two metrics affect the architects’ ability to decouple functionality and create independent, isolated microservices in the future. At the end of the day, that is what forms the overall technical debt. Using the average of these three metrics, we can further prioritize applications in the portfolio. This provides insights into immediately actionable areas, such as our next metric.

The process of modernizing takes time and money and, unfortunately, is a high-risk and often unsuccessful project. To make this process successful, it is first necessary to understand the cost of innovation, the complexity of the application and the risks involved. With these metrics at their disposal, R&D and product are well positioned for a comprehensive, data-driven conversation about modernization.

June 8, 2023
Technical Debt is Inevitable. Here’s How to Manage It

Eventually, your organization will accumulate technical debt. It doesn’t matter what programming language, software stack, computing style, host, or slick hybrid mix you use. It doesn’t matter if you’re using a waterfall or agile methodology. Ultimately, most technology becomes technical debt one day.

“Technical debt is inevitable,” said Insight’s North American chief technology officer, Juan Orlandini. “Technical debt is like credit card debt. If you’re not smart about how you use it, your balance keeps increasing.”

Today’s technological trends will one day fall out of favor, especially as talent moves on and older maintainers retire. And while this all might sound grim, acceptance is the first step toward change. Realizing this pattern of technical debt and accepting its inevitability may be the key to building future-proofed, evolvable software ecosystems.

I recently chatted with Orlandini to understand some of the significant contributors to technical debt and how software maintainers can plan for and address it. Below, we’ll outline some of the major causes of technical debt and suggest methods to tackle it and prevent it from accumulating.

Major Contributors to Technical Debt

Enterprises face growing technical debt. A study from Outsystems found that 40% of IT budgets are devoted to addressing technical debt in large enterprises. To mitigate technical debt, we must first understand the main contributors to it. The causes of technical debt are wide and varied but can be distilled into a few common areas, said Orlandini.

Unsupported software stacks. One major cause of technical debt is outdated or unmaintained software. Too often, a consumer will purchase a proprietary system that eventually becomes sunsetted, said Orlandini. Or, the vendor may evaporate during an acquisition or merger. When this occurs, the package goes unmaintained and leaves a mess on your hands.

Trends in programming languages. Another obvious area is the continual evolution of programming languages. Even today’s most popular languages aren’t that future-proofed—Python replaces Perl, Go replaces Python, Rust replaces Go and so on. The same is true for frameworks. Ruby on Rails used to be in, but now Vue.js is a more hip framework.

Aging cloud infrastructure Just because you’re in the cloud doesn’t mean you don’t have technical debt, reminds Orlandini. When early cloud adoption began, many core services we take for granted today, like authentication and authorization, weren’t offered by the cloud service providers. Some of the first-ever cloud migrations were stitched together with homegrown tooling, which may not be aging well. Or, organizations may have “lifted and shifted” legacy services into the cloud. Operators overseeing these older cloud-based environments will likely feel the pressure to refactor into more modern, cloud-native styles.

The proliferation of microservices and APIs. Furthermore, cloud services retire features all the time. Thus, it can be challenging to keep up with these minute changes to retain backward compatibility for things like API versioning shifts. In the DevOps world, web APIs are the glue that connects our tools. But in an increasingly complex landscape, it can become easy to produce shadow IT and lose track of dependencies.

Legacy physical infrastructure: It’s not just the software that becomes debt; the systems and hardware it runs on might also become obsolete. For example, the network itself might be full of old edge switches and wireless access points, says Orlandini. Or, an enterprise may hold onto aging on-premises servers. (This is often due to the depreciation benefits of using a capital expense accounting model as opposed to an operational expense model.)

Other factors that could contribute to technical debt are things like rushed decision-making, skills and resource gaps and failure to implement DevSecOps.

Ways to Tackle Technical Debt

Technical debt is a threat to innovation, so how can we mitigate it? Well, if you don’t already do so, it’s a good idea to build technical debt into your budgeting, planning and ongoing operations, said Orlandini. “You have to manage it, expect it and be responsible with your technical stacks in the same way you are responsible with your financial stacks,” he said. Here are a few other ways to manage the debt you have and avoid accumulating more.

Consider using AI to refactor legacy code. Generative AI could be leveraged to reactor legacy code into more modern programming languages. This could help automatically convert PEARL code, for instance, into JavaScript. Today’s large language models (LLMs) could help solve many of today’s problems. However, since they are built on a pre-existing body of work, they will use less trendy languages and might cause some technical debt in the process, cautioned Orlandini.

Don’t over-rely on new DevOps processes as a cure-all. DevOps can accelerate the time to release features, but it does not, by its nature, eliminate technology changes, said Orlandini. Just because DevOps helps deploy an aging framework doesn’t mean it will maintain or evolve the debt. As frameworks go out of style, IT teams must continually assess their software stack.

Remain lean whenever possible. Preventing the accumulation of unnecessary technology should be top of mind. In the DevOps world, there is a proliferation of individual services, and it’s easy to loop in a tool because it’s “the next best thing.” While sleek tools can help quickly integrate new functionality, looping in too much unnecessary tech could introduce sprawl concerns, said Orlandini. In general, he recommends keeping things as optimized as possible and carefully vetting the introduction of new technology. This includes carefully vetting new partners to ensure their business model works and that they themselves are future-proofed.

Continually maintain updates of dependencies. Today’s software infrastructure is largely composed, at its foundation, of open source software. And if these packages aren’t mature and well-supported, you may be sitting on an unstable house of cards. It’s a good idea to continually audit your OSS use and respond to updates in a timely manner.

Managing the Inevitable: Tech Debt

Tech debt is like credit card debt—easy to get into and tough to get out of. And just like its financial equivalent, you shouldn’t go too far into technical debt. Thankfully, we’ve covered some best practices to keep in mind that should help spot key problem areas and reduce bloat before things get out of hand.

As a final note, consider asking: How much stuff are you using and why? It’s become a trope, but the principle of KISS (keep it simple, stupid) bears repeating in software architecture dialogues. Because if you reduce the surface area, it will make ongoing maintenance easier and reduce the potential of cyberattacks.

The subject of technical debt is a multi-faceted issue with many factors to consider. For related insights, you can also watch the conversation between Techstrong Group Chief Content Officer Mike Vizard and Juan Orlandini on Techstrong.tv here:

June 8, 2023