devopsy.kinsta.cloud

Category: DevOps Culture

  • The Case For an API-First SCM

    The Case For an API-First SCM

    Somewhere around a decade ago, about the time DevOps really started to manifest, we began seeing technology companies—including some of the largest ones today—investing heavily in their APIs.  

    API-first companies managed to rapidly gain widespread adoption because of the benefits of being API-driven. Everything from automation and scalability to flexibility, performance, speed and monetization.  Once DevOps thinking made it increasingly clear that automation would solve problems like human error and increase performance and velocity, creating more machine-to-machine interaction became a critical factor in software delivery—because the problem in the technology world was not innovation, but rather integration.

    Twilio is a prime example. The company became famous for its API-driven approach to customer management and even had a famous Silicon Valley billboard trumpeting their success. They changed the game for customer communication for some of the largest companies in the industry, from Netflix to DoorDash to Coca-Cola, by making communication completely invisible to users. These APIs largely served developers by unifying communication on the backend and providing powerful customer-facing capabilities on the business side. But they’re not the only ones.

    API-first SCM

    API-driven companies managed to disrupt entire markets nobody imagined could be disrupted, such as Stripe for payment processing and Slack for immediate communication (their former head of developer relations, Amir Shevat, said that Slack’s developers were literally measured by their API’s active tokens). Companies that harnessed the power of their APIs saw unprecedented growth over the last decade due to the integration power they unleashed. 

    Today, there really is no self-respecting dev-first technology that doesn’t invest heavily in the robustness of its APIs. There are even entire companies today built around API integrations—from marketplaces like Rapid API to security companies like Akita Software to companies like Kong that focus solely on simplifying API gateways.

    The API-First Dev Tool Revolution

    The same transformation that disrupted various business industries is now beginning to take center stage in the technology and dev tools space. This is also part of the developer experience revolution that is the stimulus for all the nifty dev tools in the first place. As developers become the new decision-makers—masters of their own domain—and are no longer being forced to adopt and use technology someone else purchased for them, the experience of using these tools impacts their choice of whether or not to add these tools to their chain. This means that those creating the tools today need to deeply understand the developer’s workflow and optimize their products for this user persona.

    This very same evolution that typical tooling underwent is now impacting age-old tooling in the developer ecosystem—and developer pain and friction are starting to come to the forefront. This includes the need for better and more robust API capabilities, “as-code” (infrastructure-as-code, testing and security) tooling, low-code/no-code and code variations and much more.  

    It’s interesting to see that, even with the cloud-native and DevOps revolution, our most core developer tool—our source code management—hasn’t really changed much in the last 20 years. Git remains ubiquitous, but its design is optimized for developer workflows from two decades ago. It’s long past time for an API-first SCM, and there are a number of use cases that can benefit from better APIs.

    Evolution and Anatomy of APIs to be Git-Native

    Even with the most exciting innovations we’re seeing from Git SaaS tools like GitHub and GitLab—from Co-Pilot to Codespaces, pipelines and more—they’re only as strong as the technology they are built upon. As the world evolves, this fundamental weakness is proving to be a sort of kryptonite.  

    Despite Git being an old-school, monolithic tool with proprietary protocols, Git hosting providers like GitHub, GitLab and Bitbucket have invested a lot into making their services API-driven to cater to tool integration. True, each has its own API, which drove some players integrating with the service (especially in the early days) to integrate via an SDK rather than an API.

    GitHub itself provides a variety of APIs—from the simpler REST through the more elaborate GraphQL. These APIs provide you with the ability to do pretty much anything, but are somewhat cumbersome for some use cases. To deal with the core Git functionality, you are required to directly manipulate objects in the Git database.

    Code Deployment and GitOps

    As engineering organizations grow, the need for greater control of code and infrastructure resources has become a real pain—and this is where GitOps provided the methods and the tools to support automated policy management at scale. This automated policy and governance is largely enabled, machine to machine, via APIs, and many of its controls are built on code and configuration files that are hosted via Git (hence its name: Git Operations).

    When using this methodology, a configuration change is first committed to Git (as the single source of truth that will apply changes in all the relevant places). Only then does this trigger automatic processes to apply these changes universally. In some cases, these automatic processes have to commit files to Git via API.

    Another area where SCM APIs play a key role is versioning and release management.  DevOps teams need to manage the release of new versions of their applications, including creating release notes, tagging releases and creating release branches. These processes, which sometimes take place multiple times every day, include many steps and require accuracy—a comprehensive suite of efficient APIs is critical for such an operation.

    UX Customization

    Today, there’s a huge diversity of stakeholders in modern engineering processes, particularly in the context of game development, but this is also true in any kind of web-based or UI-based engineering, as well. Today, product and design are an integral part of engineering processes, but Git still remains the domain of developers.

    There are times when you’d like to have a unified interface for your asset management that pulls data from multiple APIs and sources—such as Git for source control and Google Drive for large files. The added benefit is to abstract and decouple the underlying technology from the user experience and interface so that users can be free to migrate and move between their technologies of choice. This is true whether at the SCM or the storage level without having to change the interface that the team is accustomed to.

    APIs that efficiently perform the full set of SCM functionality allow developers to use a single backend for managing code versions while still providing end users with the desired experience, regardless of what that might be.

    Versioning-as-a-Service

    SaaS products have made it significantly easier to manage and maintain our services through simple UIs and dashboards. However, this simplicity also has a tradeoff. Have you ever made a simple configuration or code change, quickly regretted doing so and then attempted to revert it via your SaaS console? In most SaaS apps that aren’t managed as code, this happens all the time—from Okta to Datadog and many others. Without the benefits of “as-code” and the version control that comes with it, reverting changes is nearly impossible.

    The most you can achieve is “undoing” a previous configuration—if you can still catch it with a CTRL+Z—simply because engineering didn’t bother (or did not have the resources) to implement versioning. Imagine if this could come as a service for all of your SaaS applications!

    For software developers, API-based version control will unlock the possibility of easily incorporating rich version control capabilities into their product—in the same way they can currently use other capabilities such as database, file storage, etc. 

    The World is Ready for an API-First SCM

    TL;DR: API-first models have delivered some of the greatest innovations that ‘dinosaur’ industries—from payments to communications—needed to move faster and do so securely. It is almost laughable that the technology industry has inherent limitations due to a core technology that has a legacy design. This should not be a technology limitation we should accept.  

    Our source code should be able to be controlled, manipulated and transformed via API to remove human toil from places it isn’t really needed and free people to focus on higher-order problems.  With the widespread adoption of generative AI and smarter machine overlords, we’ve discovered that there are many formerly manual tasks that we can offload to machines. Free SCM APIs to unlock the next generation of innovation—because soon there will be more machines than humans to manage them. APIs are the key to tapping into this power. 

  • 92% of Devs Use AI, Survey Says ¦ Intel One Mono Font

    92% of Devs Use AI, Survey Says ¦ Intel One Mono Font

    Welcome to The Long View—where we peruse the news of the week and strip it to the essentials. Let’s work out what really matters.

    This week: GitHub touts unbelievable AI stats, and Intel’s beautiful open source typeface. (more…)

  • How Open Source Can Enhance Developer Experience

    How Open Source Can Enhance Developer Experience

    Developer experience (DX) is becoming more and more of a competitive advantage among developer-facing tools and platforms. A great developer experience that’s easy to follow and implement can truly help a software tool stand out from the crowd.

    Simultaneously, open source software is all the rage. In fact, a recent StackOverflow survey found open source software (OSS) to be, by far, the most revered and proven technology trend.

    With that in mind, what are the synergies between developer experience and open source? And how can building in an open source format aid the developer experience for your public-facing toolsets and internal development platforms?

    I recently met with Adam Frank, SVP of product and marketing at Armory, to explore the state of developer experience and open source adoption. According to Frank, open source can bring numerous DX benefits. Below, we’ll explore these areas and consider why organizations might benefit from adopting more open source within their developer-facing initiatives.

    The State of Developer Experience and Open Source

    So, what is the state of developer experience, and why is it important now? Well, DX is continually leveraged to help improve agility and competitiveness in the market. In addition to granting differentiation, DX also enables more flexibility, helping users switch from one tool to another if necessary.

    “Developer experience is at the forefront right now because technology and change are moving faster than ever before,” said Frank. “Companies need to focus on DX as a competitive advantage to survive and thrive.” At the same time, open source can help organizations move quickly and iterate more efficiently.

    Yet, historically, open source has not been associated with fantastic DX. Most of us have at least one horror story of an open source project simply going belly-up without warning. It’s common for an OSS tool to lack comprehensive documentationor suffer from a lack of funding for ongoing maintenance, zero customer support and deprecated modules. Some tools that developers directly download and contribute to may not possess a quality DX emphasis, says Frank.

    However, the status quo for OSS has undoubtedly evolved over the years, he adds. Nowadays, there are more established groups, like Linux Foundation’s Cloud Native Computing Foundation (CNCF), with specific Technical Oversight Committees (TOCs) to oversee individual projects. The maturity of OSS ownership now ensures core projects stay relevant and receive continual support—not to mention ongoing developer experience enhancements.

    On the other side, OSS is often absorbed by a platform team to enable their internal developers to move faster or ship more code, says Frank. From this perspective, open source communities give platform engineering the opportunity to contribute back to the project, helping them solve deeper-level problems that are more specific to their environments.

    How Open Source Can Enhance Developer Experience

    Not all IT leaders are sold on the idea of building in a transparent, open source model. Yet, this enhanced transparency can bring dividends in the long run, especially where developer experience is considered. Frank shared some of the benefits of adopting open source.

    Receiving Feedback From The Community

    “As a product manager, you want as much feedback from your users as you can gather,” said Frank. Receiving user feedback is a great way to fix DX issues, and OSS gives you that on a larger scale. By adopting open source, you benefit from collecting insights from your customer base and the developer community at large. This can translate into a broader range of use cases, which can significantly improve projects and overall experience.

    Increasing Ecosystem Transparency

    Building in a more transparent and open fashion so that everyone has the ability to submit pull requests can also help you work together with potential partners. This makes it easier to develop third-party integrations that ensure the broader ecosystem can work together. By decreasing the barrier to entry through OSS, you provide others with the ability to solve more problems around developer experience.

    Streamlining the Developer Toolchain

    “Developers want to innovate; they want to solve problems,” said Frank. Open source can also help expose inefficiencies in the developer toolchain and, notably, many open source CI/CD tools are often leveraged to automate deployment and increase efficiency. The end result is that developers can focus on innovating their own competitive advantage. “Making the deployment aspect of the developer lifecycle really seamless can significantly improve their experience,” he said. Therefore, any tools or extensions around CI/CD are an excellent area to consider open-sourcing.

    Increasing Flexibility

    Another benefit of the open source model is that you have far more flexibility than off-the-shelf proprietary software. As a customer, you don’t have to wait to submit enhancement requests. With OSS, you can fork a project or quickly collaborate with the maintainers. This equates to making fixes far quicker and providing the ability to customize developer experiences to the tasks at hand.

    Best Practices to Ensure Open Source Improves DX

    Open source provides many key benefits that can enhance software development and delivery. But, this should be taken with a grain of salt, as best practices still must be followed to ensure open source enhances, not hinders, developer experience.

    For example, Frank mentions that simplicity should be a guiding light. “Have simplicity for the user experience design in the forefront of everything you’re building and thinking about,” said Frank. Adopting simplicity should help ensure developer-facing materials are structured well and documented in an easy-to-comprehend manner.

    On that note, open source projects will require standards around areas like good documentation and well-written APIs that abide by contracts, said Frank. Anything less will hinder developers and confuse them with disparate data formats. On that note, he has witnessed OpenTelemetry emerge as a common way to unify data that systems produce with standard formatting, which increases the observability of these environments.

    Lastly, it’s critical to hire talent with experience in open source. Bringing in folks that have knowledge of how OSS communities operate will make onboarding easier and encourage safer open source security practices as well.

    What the Future of DX and OSS Holds

    Open source appears to aid developer experience in many ways, from driving more innovation, opening software up to feedback and developing a closer-nit developer community. But what does the future of developer experience and open source look like?

    According to Frank, we will likely see increased data standardization that makes it easy to work with various projects and data formats. OpenTelemetry, for example, makes aggregating logging easier. We’ve also seen standardization around OpenAPI Specification for describing web APIs. Another example is FinOps Foundation’s FOCUS specification to unify cost analysis.

    Open source tends to leverage open source. And as the industry continues to coalesce around open standards, we’ll likely see more robust, compatible layers of abstraction emerge throughout the market, Frank predicts.

  • Grafana Labs Slashes Time to Create Observability Dashboards

    Grafana Labs Slashes Time to Create Observability Dashboards

    At its GrafanaCON 2023 event today, Grafana Labs updated its core platform for visualizing data which makes it simpler to set up dashboards in minutes. The updates are part of a celebration of the 10th anniversary of a tool that is now widely used by DevOps teams.

    Ryan McKinley, a distinguished engineer for Grafana Labs, said Grafana 10 should help make DevOps more accessible to a wider range of practitioners with limited experience working with observability tools. The latest release also makes it simpler to share knowledge with external stakeholders via public dashboards, he added.

    In addition, Grafana 10 adds expanded role-based access control (RBAC) capabilities and streamlined implementation of the secure access markup language (SAML) to ensure authentication.

    Grafana Cloud has also been updated to provide a secure way to connect to data residing in an on-premises IT environment or in a cloud with the observability platform the company provides via a software-as-a-service (SaaS) platform.

    Finally, Grafana Labs is also making the engine it created to visualize data more accessible to third-party developers that are embedding visualizations based on Grafana within their applications, noted McKinley.

    Grafana Labs

    There are now more than 150 plugins for various data sources that have been made available to more than 20 million users of the Grafana visualization framework. Not all those users are working in DevOps environments, but Grafana is widely used to enable observability across increasingly complex IT environments. The Grafana Cloud seeks to capitalize on that popularity by adding a service that uses an instance of open source Grafana Mimir software to store data collected in a format originally defined for the open source Prometheus monitoring tool.

    It’s still early days as far as adoption of observability is concerned, but it’s apparent that as application environments become more complex, the plethora of monitoring tools that DevOps teams rely on will need to be streamlined. Observability platforms promise to unify logs, metrics and traces to simplify launching queries to identify the root cause of an issue.

    The rate at which DevOps teams will embrace observability will naturally vary, but the biggest obstacle might not be the platforms themselves. Instead, the issue is understanding what queries can help DevOps teams better understand the root cause of an IT issue before there is a major disruption. In the long term, it’s expected that machine learning algorithms will leverage the data collected by observability platforms to automatically identify issues that might lead to a disruption long before it actually occurs.

    Of course, the result of any query that gets launched needs to be visualized in a way that is easily understood. Grafana Labs is now making a case for a more integrated approach that, at its core, employs its visualization tools to make it simpler to discern what’s occurring in IT environments. After all, no matter how talented any DevOps team may be, it’s still not possible to manage what was never seen in the first place.

  • Avoiding the Pain of a ‘Resume-Driven Architecture’

    Avoiding the Pain of a ‘Resume-Driven Architecture’

    Successful new technologies follow a predictable arc, from initial innovation to technical gold rush and, ultimately, to mass market adoption. As a consequence, they achieve strong standardization and productization. Being highly attuned to where a specific field lies on the maturity arc is how you yield the greatest benefits to a business. But what happens if your developers are not thinking about this, too? You end up with what I call a “resume-driven architecture,” and it can be highly costly to your business.

    A resume-driven architecture occurs when the interests of developers lead them to designs that no longer align with maximized impacts and outcomes for the organization. Often, the developer clings to a technology that provides them a greater level of control and, at least initially, a higher salary. Meanwhile, the organization gets an architecture that only a handful of people know how to manage and maintain, limiting the available talent pool and hindering future innovation.

    To understand why this happens and how to avoid it, it’s helpful to look at this technology arc in more detail. It’s a pattern that has played out up and down the stack, with technologies like big data platforms (now dominated by Snowflake and Databricks), Kubernetes and even the broader move to the cloud. Failing to grasp this concept of a resume-driven architecture can increase costs and hold back innovation.

    Here’s how this technology arc typically plays out, using Apache Spark as an example:

    • Stage One: A technology breakthrough like Spark emerges, providing a significant advantage to early movers.
    • Stage Two: Other companies see the benefits, and the technology receives widespread interest. Specialists are in high demand and command big salaries. Many adjacent professions naturally invest in becoming Spark experts.
    • Stage Three: The technology matures and reaches a level of standardization with an ecosystem of tools and services around it. Specialized knowledge, such as how to tune your Spark job parameters, becomes less valuable.
    • Stage Four: Mass convergence, productization and simplification occurs. As examples, Databricks becomes the primary Spark vendor and launches SQL warehouses for easier data processing. Snowflake launches Snowpark for Python as a simpler model than classic PySpark jobs. Nobody worries about tuning most parameters anymore as the underlying technology becomes part of the stack: Powerful but almost invisible.

    It’s important for businesses to take advantage of this productization stage when it comes along. There’s no sense in investing resources in a bespoke architecture if it’s not providing you with any differentiation—especially when competitors are achieving the same outcome with fewer resources.

    Moreover, getting stuck in a Stage Two mindset when the field moves on to Stage Three (or, worse, Stage Four) and cuts you off from the next wave of innovation. Subsequent technology breakthroughs often build on top of—and interoperate with—the previous technology layers. If you’re stuck with a custom architecture when the industry has moved on, you can miss out on the next wave of innovation and fall further behind competitors.

    The problem is, developers who’ve made a great career from implementing custom architectures are often reluctant to move on. After all, mastery of that previous innovation is what got them to where they are. They may not have realized that the world has progressed around them, or they may worry a standardized architecture with managed services and off-the-shelf products undermines their power and autonomy. This is how organizations fall victim to a resume-driven architecture. I’ve observed this several times when speaking with engineering leaders, and it’s become more common with the rapid innovation happening around data.

    As engineers, our value is in building things and, ideally, in solving innovative, artisanal problems. But this identity and desire can confuse what we want from our own jobs with what our team needs from us. When standard patterns and standard products emerge, we need to adopt them because that’s the way to maximize our impact and ultimately free our time and energy to move further “up stack” and have an even greater impact.

    Developers who would rather stick to a custom architecture often argue that they can squeeze out greater performance, but this is rarely a justification. You may end up with an app that’s 10% faster, but rarely does that come close to making up for the time and energy invested, the loss of flexibility and, ultimately, being held hostage by your own technology stack.

    So how does an organization avoid the trap of a resume-driven architecture? Strong leadership is key. Look critically at the business and ask which areas of your stack are truly differentiators because it’s only those areas that may—just may—justify a unique architecture. If you do go that route, it needs to provide huge benefits to the business to be worthwhile.

    For example, “data” is not in itself strategic to you; every company is now a data company, and the vast majority of things you do with it—how you source, store, process and distribute data—is no different from what everyone else is doing. What does matter are the data products that fuel your business and, most importantly, how fast you can build them.

    That’s why, as a developer or engineer, bespoke stacks are not your friend. Instead, architect your platform in a way that allows other developers to onboard quickly; that allows you to upgrade and swap out components easily over time and that doesn’t leave you held hostage to a developer whose main interest is retaining their value by sticking with an unnecessarily complex architecture.

  • Leading DevOps Transformations With AI

    Leading DevOps Transformations With AI

    In my recent article Revolutionizing the Nine Pillars of DevOps with AI-Engineered Tools, I explained that AI-engineered tools can help with automating repetitive tasks, improving decision-making with predictive insights and facilitating proactive issue resolution. AI can drive a higher level of quality, security and performance in the software life cycle. Leadership, the first of the nine pillars of DevOps, is the catalyst for a successful DevOps transformation using AI-engineered tools. It’s the guiding force that brings about the alignment of people, processes and technology in the pursuit of continuous delivery and operational excellence.

    How AI-Engineered Tools Assist DevOps Leaders

    Vision and Direction: Leaders define the vision and strategic direction for a DevOps transformation. They set expectations, define goals and align all stakeholders around a common purpose. By clearly articulating the benefits and objectives of the transformation, leaders can secure buy-in from the whole organization and guide their teams toward success. AI can provide data-driven insights to assist in strategic planning and decision-making. By analyzing historical data and current trends, AI can help leaders make informed predictions about the future and set the direction accordingly.

    Cultural Change: DevOps is not just about technology, it’s about changing organizational culture. Leaders are instrumental in driving this change. They model the collaborative, open and learning-focused culture that DevOps requires. They facilitate communication and collaboration across traditionally siloed teams and encourage a mindset of shared responsibility for product quality and customer satisfaction. AI tools can facilitate collaboration and knowledge sharing, helping to break down silos. These tools can enhance communication, support shared workspaces, and even analyze team dynamics and communication patterns to provide suggestions for improvement.

    Resource Allocation: Effective leaders ensure teams have the necessary resources, including tools, training and time to successfully adopt DevOps practices. They advocate for their teams and prioritize investments in areas that will maximize the benefits of DevOps, such as automation tools or cloud services. AI can assist in optimizing resource management. Predictive analytics can help leaders identify where investments will yield the greatest return, and automation can free up team members’ time to focus on high-value tasks.

    Risk Management: Leaders play a key role in managing the risks associated with a DevOps transformation. They set the pace of change, balancing the need for speed with the need to maintain stability and security. They make tough decisions about when and how to deploy new practices or tools and they are prepared to deal with setbacks or failures as opportunities for learning and improvement. AI can play a crucial role in identifying and mitigating risks. By predicting potential issues in the development and operations pipelines, leaders can take preventative action. AI can also improve incident management, speeding up the detection and resolution of issues.

    Continuous Improvement: The nature of DevOps is iterative and based on continuous feedback and improvement. Leaders need to instill this principle within their teams, emphasizing the importance of regular reflection, learning, and adaptation. Leaders can drive ongoing improvement and innovation in their DevOps practices by fostering an environment where experimentation is encouraged and failure is seen as a learning opportunity. AI can support continuous improvement by analyzing performance data and identifying bottlenecks or areas for improvement. AI-powered analytics can turn raw data into actionable insights, helping leaders understand the impact of their actions and make informed decisions about future changes.

    Risks of Not Leading With AI-Engineered Tools

    In an increasingly digital world, the use of AI-engineered tools is becoming standard practice in DevOps. While every organization must assess its own readiness and capacity to adopt AI, the risks of not doing so are significant and could impact competitiveness and operational efficiency.

    • Missed Efficiency Gains: AI offers substantial efficiency improvements through automation, prediction and analytics. Organizations not leveraging these benefits could find themselves outpaced by competitors that do.
    • Decreased Competitiveness: AI can enable faster and more reliable software delivery, quicker response to market changes and more efficient problem-solving. Without AI, organizations might struggle to keep up with competitors in terms of innovation and speed to market.
    • Inferior Decision Making: AI enhances decision-making by providing leaders with data-driven insights. Without these insights, leaders might make less informed decisions, potentially resulting in suboptimal outcomes.
    • Reduced Agility: AI can help organizations anticipate and quickly respond to changes. Not using AI may limit an organization’s ability to adapt quickly to new market trends or customer demands.
    • Increased Operational Costs: AI can significantly reduce operational costs by automating repetitive tasks, predicting system failures and optimizing resource usage. Without AI, organizations may face higher operational costs.

    Transformation Roadmap With AI-Engineered DevOps

    The successful integration of AI-engineered tools into DevOps is not a one-time project but a continuous journey of learning, experimenting and adapting. Leaders play a crucial role in guiding their organizations on this journey, and developing their own AI competencies is a key part of that.

    1. Education and Awareness: Begin with enhancing your understanding of AI fundamentals, its capabilities, limitations and implications. This can be done through formal training, online courses, seminars, workshops or self-study.
    2. Identify AI Opportunities: Leaders need to identify areas in their DevOps practices where AI can provide the most benefit. This might be where automation could save time, where predictive analytics could provide valuable insights or where machine learning could improve system performance.
    3. Start Small and Experiment: Begin by piloting AI tools in one or two areas. This provides an opportunity to learn about AI’s capabilities and challenges in a controlled, low-risk environment.
    4. Gather and Analyze Data: Data is crucial for AI. Leaders must ensure their organizations have systems in place to collect, clean and manage data. They also need to learn how to interpret the data and insights provided by AI tools.
    5. Build a Multidisciplinary Team: AI requires various skills, including data science, software engineering and domain expertise. Leaders should aim to build diverse teams and foster a culture of collaboration between these different skill sets.
    6. Scale and Integrate: Once they have seen success with initial pilot projects, leaders can start to scale their use of AI tools and integrate them more deeply into their DevOps practices.
    7. Continuously Learn and Adapt: As with any technology, the AI landscape is constantly evolving. Leaders need to commit to continuous learning and stay updated with the latest trends, tools and best practices.
    8. Develop Ethical and Responsible AI Practices: Leaders should ensure AI is used responsibly within their organization, considering aspects like data privacy, transparency and bias. They should create guidelines and practices for ethical AI use.

    Pitfalls and Challenges

    By being aware of potential pitfalls and planning for them in advance, leaders can increase the odds of a successful implementation of AI-engineered tools in their DevOps practices.

    Resistance to Change: Implementing AI tools often requires significant changes to workflows and practices. This can be met with resistance from team members. Effective change management practices can help overcome this resistance. Communicate the benefits of the change, provide ample training and support and involve team members in the process.

    Data Privacy and Security Concerns: AI tools often rely on access to sensitive data, which can raise privacy and security concerns. Develop and adhere to strict data management policies, ensuring compliance with all relevant privacy and security regulations. Use secure AI tools that offer robust data protection features.

    Skills Gaps: Implementing AI tools often requires skills that existing team members may not have. Invest in training for your team, hire new staff with the necessary skills or bring in consultants or external experts to help.

    High Expectations: AI is often perceived as a magic bullet that will solve all problems. This can lead to unrealistic expectations and eventual disappointment. Set realistic expectations about what AI can and cannot do. Communicate clearly that AI is a tool that can enhance efficiency and effectiveness, but it requires human oversight and cannot replace all human tasks.

    Choosing the Wrong Tools: With so many AI tools available, it’s easy to choose ones that aren’t a good fit for your organization’s needs. Carefully evaluate potential tools based on your specific needs and constraints. Look at factors like compatibility with existing systems, ease of use, scalability and vendor support.

    Lack of a Clear Strategy: Implementing AI tools without a clear strategy can lead to disjointed efforts and wasted resources. Develop a clear strategy and roadmap for your AI implementation. This should outline your goals, the steps to achieve them and how you will measure success.

    Summary

    Leadership plays a crucial role in the successful transformation and adoption of AI-engineered tools in DevOps practices. The journey involves educating themselves about AI, identifying suitable opportunities, experimenting with AI tools, gathering and analyzing data, building multidisciplinary teams, scaling AI usage and promoting ethical AI practices. However, challenges like resistance to change, data privacy and security concerns, skill gaps, high expectations and selecting the right tools can be encountered. These can be mitigated with effective change management, strict data policies, training and hiring strategies, setting realistic expectations, careful tool evaluation and a clear AI implementation strategy.

  • Five Great DevOps Job Opportunities

    Five Great DevOps Job Opportunities

    DevOps.com is now providing a weekly DevOps jobs report through which opportunities for DevOps professionals will be highlighted to better serve our audience.

    Our goal in these challenging economic times is to make it easier for DevOps professionals to advance their careers.

    Of course, the pool of available DevOps talent is still relatively constrained, so when one DevOps professional takes on a new role it tends to create an opportunity for others.

    The five job postings shared this week are selected based on the company looking to hire, the vertical industry segment and, naturally, the pay scale being offered.

    We’re also committed to providing additional insights into the state of the DevOps job market. In the meantime, for your consideration:

    LinkedIn

    Underdog.io
    New York, New York
    DevOps Engineer
    $114,000 to $206,000

    SimplyHired.com

    Verizon
    Lone Tree, Colorado
    DevOps Engineer
    $107,000 to $226,000

    Indeed.com

    Erie Insurance
    Erie, Pennsylvania
    DevOps Solutions Architect
    $104,838 to $167, 468

    Dice.com

    Nityo Infotech Corp
    Mountain View, California
    DevOps Engineer – AWS Cloud
    $100,000 to $140,000

    CareerBuilder.com

    Boston College
    Chestnut Hill, Massachusetts
    Senior DevOps Systems Administrator
    $96,000 to $120,000

  • The API Life Cycle Mixer’s Manual: What Goes Into a Single Source of Truth?

    The API Life Cycle Mixer’s Manual: What Goes Into a Single Source of Truth?

    In bartending, the mixer’s manual is a trusted source of information. These books can contain hundreds of recipes for classic cocktails, some of which might be 100 years old, while others might be for drinks that only recently arrived on the scene. Mixer’s manuals also can be great sources of inspiration for new, yet-to-be-tested recipes for exciting additions to a bar’s or restaurant’s offerings. This balancing act of preserving the old while encouraging exploration into the new can provide a valuable lesson for API development in the software world. An API’s “single source of truth” is, in many ways, the software organization’s own mixer’s manual. This is especially true when the right approach is taken to ensure that software remains functional and performant and serves up experiences that delight customers and keep them coming back for more.

    The Rise of APIs and Mixology

    While we might think of mixology and the mixologists who practice this craft as relatively new terms, the first recorded use of the word “mixologist” dates back to 1852 and “mixology” followed shortly after.

    Just how similar are the work, mindsets and passions of mixologists and API developers and designers? Check out this definition of a mixologist and count the similarities for yourself:

    “The term ‘mixologist’ refers to someone who studies the history of mixed drinks, has a rich appreciation of the ingredients and techniques used and regularly creates new and innovative mixed drinks … their job title implies that they do a significant portion of their job behind the scenes, creating new craft cocktails and putting their signature twist on existing favorites.”

    The best API developers, designers and architects of the world should recognize a lot of the above in their own work. They have deep respect for their craft and are always honing it further. They seek to understand the tastes and palates of their consumers. They factor in other stakeholders’ inputs, they test, iterate and test their creations again before “serving” those APIs to production.

    Like mixologists and bartenders (if you see them as two different roles), every professional along the API life cycle should understand the needs of consumers, customers and the business–and then design and develop their APIs accordingly. An essential part of this process is continuous validation and adaptation to market requirements.

    Maintaining Confidence in Ever-Changing Recipes

    Changes to APIs, even minor ones, can have significant downstream effects. And the same can be said for tweaks made to an age-old cocktail recipe, where adding or removing a single dash of this or that can completely change the flavor of a drink … as well as people’s interest in it.

    While there’s no “standard” rate of change to APIs to plan for, you can expect changes to be needed with each new software release. This could mean changes are being made (and need to be tested against and validated) every few weeks, every month or even every day, depending on your release cadence. There also can be as-needed, ad-hoc changes.

    What’s crucial for teams to ensure is that these changes do not disrupt an API’s functionality or performance. To make this possible, teams are increasingly leveraging API contract testing to provide a safety net of sorts. This comes in the form of testing that validates changes against a “contract” that details an API’s original, agreed upon and, therefore, required functionality.

    The ability to foresee the impacts of changes to APIs can depend on the maturity of an engineering team and its ability to collaborate across departments, time zones and varying levels of technical expertise. Tools and guidelines that promote standardization and collaboration can prevent mishaps from overlooking or being blind to negative downstream effects.

    Documentation is Key

    Whether we’re talking about being able to trust a single source of truth for the most up-to-date version of a published API or the latest edition of a mixer’s manual for tried-and-true drink recipes, clear and concise documentation is key. But it’s also not enough for this documentation to simply exist.

    It seems like common sense—although you’ve probably worked somewhere where it wasn’t—that your API documentation also needs to be easily accessible, understandable and usable. Ensuring all of these things leads to smoother onboarding and a positive developer experience and, ultimately, to wider adoption of a functional, performant API that meets or even exceeds expectations.

    API documentation is often the first point of contact a developer has with an API. It serves as a guide, walking developers through the API’s features, endpoints, request/response models and error messages. It also provides insight into the underlying business logic and unique characteristics of an API.

    Good API documentation follows the principle of a single source of truth, meaning that it should be comprehensive, up-to-date, and consistent. It should always mirror the actual behavior of the API, reducing the guesswork for developers and minimizing the potential for misunderstandings and errors.

    Just as a mixologist relies on a detailed and accurate cocktail recipe to deliver a pleasing experience to a patron, API developers depend on good documentation to create and deliver useful, value-add APIs and powerful applications.

    Continuous Improvement = Continuous Education

    Like a mixologist who meticulously adjusts the ingredients of a cocktail or who creates new cocktails to meet the evolving tastes of customers, developers must constantly adapt their APIs to meet changing market demands, business needs and industry standards.

    This is achieved through a potent blend of market research, collaboration, adherence to standards and governance and meticulous testing. However, as software development, testing and deployment technologies and processes continue to evolve, the tech industry must continue educating itself and refining its practices.

    With a mindset focused on constant learning and continuous improvement and a dependable single source of truth as their guide, development teams can confidently serve up exceptional API experiences in every release. In the end, the aim is to create APIs that are reliable and easy to consume and that meet the unique needs of the moment–much like a mixologist’s perfect beverage.

  • Technical Debt is Inevitable. Here’s How to Manage It

    Technical Debt is Inevitable. Here’s How to Manage It

    Eventually, your organization will accumulate technical debt. It doesn’t matter what programming language, software stack, computing style, host, or slick hybrid mix you use. It doesn’t matter if you’re using a waterfall or agile methodology. Ultimately, most technology becomes technical debt one day.

    “Technical debt is inevitable,” said Insight’s North American chief technology officer, Juan Orlandini. “Technical debt is like credit card debt. If you’re not smart about how you use it, your balance keeps increasing.”

    Today’s technological trends will one day fall out of favor, especially as talent moves on and older maintainers retire. And while this all might sound grim, acceptance is the first step toward change. Realizing this pattern of technical debt and accepting its inevitability may be the key to building future-proofed, evolvable software ecosystems.

    I recently chatted with Orlandini to understand some of the significant contributors to technical debt and how software maintainers can plan for and address it. Below, we’ll outline some of the major causes of technical debt and suggest methods to tackle it and prevent it from accumulating.

    Major Contributors to Technical Debt

    Enterprises face growing technical debt. A study from Outsystems found that 40% of IT budgets are devoted to addressing technical debt in large enterprises. To mitigate technical debt, we must first understand the main contributors to it. The causes of technical debt are wide and varied but can be distilled into a few common areas, said Orlandini.

    Unsupported software stacks. One major cause of technical debt is outdated or unmaintained software. Too often, a consumer will purchase a proprietary system that eventually becomes sunsetted, said Orlandini. Or, the vendor may evaporate during an acquisition or merger. When this occurs, the package goes unmaintained and leaves a mess on your hands.

    Trends in programming languages. Another obvious area is the continual evolution of programming languages. Even today’s most popular languages aren’t that future-proofed—Python replaces Perl, Go replaces Python, Rust replaces Go and so on. The same is true for frameworks. Ruby on Rails used to be in, but now Vue.js is a more hip framework.

    Aging cloud infrastructure Just because you’re in the cloud doesn’t mean you don’t have technical debt, reminds Orlandini. When early cloud adoption began, many core services we take for granted today, like authentication and authorization, weren’t offered by the cloud service providers. Some of the first-ever cloud migrations were stitched together with homegrown tooling, which may not be aging well. Or, organizations may have “lifted and shifted” legacy services into the cloud. Operators overseeing these older cloud-based environments will likely feel the pressure to refactor into more modern, cloud-native styles.

    The proliferation of microservices and APIs. Furthermore, cloud services retire features all the time. Thus, it can be challenging to keep up with these minute changes to retain backward compatibility for things like API versioning shifts. In the DevOps world, web APIs are the glue that connects our tools. But in an increasingly complex landscape, it can become easy to produce shadow IT and lose track of dependencies.

    Legacy physical infrastructure: It’s not just the software that becomes debt; the systems and hardware it runs on might also become obsolete. For example, the network itself might be full of old edge switches and wireless access points, says Orlandini. Or, an enterprise may hold onto aging on-premises servers. (This is often due to the depreciation benefits of using a capital expense accounting model as opposed to an operational expense model.)

    Other factors that could contribute to technical debt are things like rushed decision-making, skills and resource gaps and failure to implement DevSecOps.

    Ways to Tackle Technical Debt

    Technical debt is a threat to innovation, so how can we mitigate it? Well, if you don’t already do so, it’s a good idea to build technical debt into your budgeting, planning and ongoing operations, said Orlandini. “You have to manage it, expect it and be responsible with your technical stacks in the same way you are responsible with your financial stacks,” he said. Here are a few other ways to manage the debt you have and avoid accumulating more.

    Consider using AI to refactor legacy code. Generative AI could be leveraged to reactor legacy code into more modern programming languages. This could help automatically convert PEARL code, for instance, into JavaScript. Today’s large language models (LLMs) could help solve many of today’s problems. However, since they are built on a pre-existing body of work, they will use less trendy languages and might cause some technical debt in the process, cautioned Orlandini.

    Don’t over-rely on new DevOps processes as a cure-all. DevOps can accelerate the time to release features, but it does not, by its nature, eliminate technology changes, said Orlandini. Just because DevOps helps deploy an aging framework doesn’t mean it will maintain or evolve the debt. As frameworks go out of style, IT teams must continually assess their software stack.

    Remain lean whenever possible. Preventing the accumulation of unnecessary technology should be top of mind. In the DevOps world, there is a proliferation of individual services, and it’s easy to loop in a tool because it’s “the next best thing.” While sleek tools can help quickly integrate new functionality, looping in too much unnecessary tech could introduce sprawl concerns, said Orlandini. In general, he recommends keeping things as optimized as possible and carefully vetting the introduction of new technology. This includes carefully vetting new partners to ensure their business model works and that they themselves are future-proofed.

    Continually maintain updates of dependencies. Today’s software infrastructure is largely composed, at its foundation, of open source software. And if these packages aren’t mature and well-supported, you may be sitting on an unstable house of cards. It’s a good idea to continually audit your OSS use and respond to updates in a timely manner.

    Managing the Inevitable: Tech Debt

    Tech debt is like credit card debt—easy to get into and tough to get out of. And just like its financial equivalent, you shouldn’t go too far into technical debt. Thankfully, we’ve covered some best practices to keep in mind that should help spot key problem areas and reduce bloat before things get out of hand.

    As a final note, consider asking: How much stuff are you using and why? It’s become a trope, but the principle of KISS (keep it simple, stupid) bears repeating in software architecture dialogues. Because if you reduce the surface area, it will make ongoing maintenance easier and reduce the potential of cyberattacks.

    The subject of technical debt is a multi-faceted issue with many factors to consider. For related insights, you can also watch the conversation between Techstrong Group Chief Content Officer Mike Vizard and Juan Orlandini on Techstrong.tv here: