devopsy.kinsta.cloud

Category: Doin’ DevOps

  • 3 Quick Fixes for Event-Driven Architecture (EDA) Complexity

    3 Quick Fixes for Event-Driven Architecture (EDA) Complexity

    Event-driven architecture (EDA) and the adoption of event streaming throughout enterprises are increasingly becoming essential architectural requirements for businesses across almost every industry. However, the more companies use it, the greater the complexity becomes. Event portals can play a pivotal role in not only gaining comprehensive insights into the intricate and expanding event-streaming ecosystem but also facilitating the auditing, management and governance of events.

    According to a recent IDC Infobrief, 93% of companies employing EDA across various applications reported that EDA either met or exceeded their expectations. Moreover, 82% of IT leaders intend to apply EDA to two to three new use cases within the next 24 months.

    Nonetheless, the number of event streaming use cases results in a surge of data volumes. This, in turn, entails the adoption of more brokers, projects and products for event streaming. For instance, an organization might use open source Kafka for one specific use case while turning to brokers like Confluent for another and Amazon MSK for yet another distinct purpose.

    While the real-time processing of data is certainly a positive development, numerous experienced EDA adopters have yet to maximize its potential. Frequently, the situation involves a one-to-one exchange, where a stream is consumed only once, or organizations, having advanced through various stages of maturity with multiple streaming use cases, find themselves entangled in a complex network of brokers, clusters, topics and overlapping schemas.

    Let’s explore three typical challenges that organizations encounter as they progress toward event-driven maturity.

    Success Needs Visibility – Make Sure Your Event Portal Can Provide It

    Application decoupling is great for runtime, but this lack of visibility on both the publish and subscribe side can cause issues when making changes to existing applications. Application producers don’t know who the consumers are and consumers don’t know who the publishers are.

    An event portal offers a single window into an event-steaming ecosystem. It provides a native discovery agent to scan, for instance, a Kafka cluster and its schema registry to produce a visual representation of every topic and its schema(s) across multiple versions–and, importantly, who its consumers are.

    If an organization were to decompose its monolithic applications into event-driven microservices that communicate with each other via, say, Apache Kafka, one of the biggest challenges is to understand and manage the infrastructure and information flows.

    Specifically, how to see the microservices affected by a given change before deploying a new feature or function to ensure it won’t bring that system down – even for as few as five seconds.

    By using an event portal, organizations can automatically scan their system and visualize a complete map of endpoints and event streams instead of manually diagraming event streams between microservices.

    Using Data at Its Best – in Real-Time

    Let’s extrapolate the poor visibility issue from one application across the entire enterprise.

    For instance, if you’ve been using Kafka for several years, especially with many departmental or specific application-type use cases, your Kafka cluster now is a treasure trove of real-time data. But data is most valuable at the moment it’s produced – as per Forrester: “Data is, without a doubt, valuable. But when stored in vaults and locked down, it is not.” Real-time data is the most valuable data that exists. But siloed event streaming data means other departments, decision-makers, customers and partners don’t know about it – therefore, it’s not getting shared or reused to its full potential.

    Typically, developers don’t have anywhere to go to find this data treasure trove. Some have resorted to building Confluence or Wiki pages that try to document that this data exists, using SharePoint or Word documents. Noble intentions, but without real-time data mining and updates, this information quickly gets stale and out of date.

    Again, this is where an event portal makes a difference, providing a perpetually up-to-date catalog of data detailing all topics, event streams, schemas and pub/sub interfaces for each application, along with owners and points of contact, as well as changes for each of the managed EDA entities. This helps expedite development by letting developers easily share, discover and re-use any existing Kafka or event streaming asset, both inside and outside the organization.

    For a real-world example of this in action, look at the Federal Aviation Administration and its SWIM (system-wide information management) infrastructure that distributes real-time information to FAA systems across the United States.

    Through secure gateways, external partners of the FAA can tap into the flow of events. Airlines and other industry partners that need the SWIM data get it in real-time, without perpetually requesting updates. Whether it’s the availability of gates at an airport, the position data of planes in the sky, or the weather for a region – external partners have the latest information without needing to ask.

    Don’t Forget to Govern Event Streams!

    The decentralized and dynamic nature of event-driven systems introduces unique security challenges. One of the common trade-offs with event streaming is that it does include access control rules, but developers can err on the side of being too permissive to ensure agility. If streams are not visible, properly cataloged, and data updated regularly, this presents problems with data security, governance and compliance.

    Using an event portal designed to give organizations visibility and control over their event streaming can turn security from reactive – to proactive. Users can organize systems into application domains, create and import payload schema definitions in a variety of formats, including AsyncAPI, define event interactions between applications and microservices, and create events and associated topic addresses using topic structure-proven practices. With the right visibility, administrators can ensure security, governance and compliance with internal policies and government regulations.

    EDA to the Rescue!

    A single, multi-broker event gateway technology offers a solution for organizations to uncover, regulate and oversee the life cycle of their real-time event streams throughout the enterprise. Its significance will continue to grow as more organizations adopt EDA as a fundamental platform.

  • Why DevOps is Key to Software Supply Chain Security

    Why DevOps is Key to Software Supply Chain Security

    DevOps is designed to increase business value and responsiveness through rapid, high-quality service delivery, made possible through fast-paced, iterative IT service delivery. The term DevSecOps emerged as a way to remind us that security was always intended to be part of the software delivery process. However, with the barrage of attacks on the software supply chain showing no signs of abating, the question now is whether even the tightest DevOps pipelines can maintain the pace-protection balance.

    It’s important to note that the term “software supply chain” has been used primarily to refer to the tools and processes used to build and deploy software applications. Now, however, with the rise of infrastructure-as-code (IaC), the term also refers to tools and processes used to build and deploy IT infrastructure.

    Meet Mandates, Reduce Risk

    Software supply chain attacks such as SUNBURST, HTTP/2 Rapid Reset or the breaches associated with the MOVEit vulnerability have fed news cycles for the last few years, and that likely won’t change anytime soon. Attacks on the software supply chain increased 742% since 2020. What we are seeing as a result are new concerns about supply chain security and DevOps requirements. This has led to confusion and often a mish-mash of tooling that may exacerbate rather than mitigate supply chain security issues.

    Arguably the most impactful salvo over the software supply chain security line was president Biden’s 2021 Executive Order 14028, which established new mandates, including the delivery of SBOMs, as minimum requirements for software used by the U.S. federal government. More recently, the FDA has mandated that “all medical devices running software must create and maintain” SBOMs. I expect to see government and industry requirements increase over time.

    These are just a couple of relatively recent examples. There are many more—and no doubt many more to come.

    So, if you’re moving at the speed of DevOps, how do you address these current and future mandates for software supply chain security, as well as the continually emerging vulnerabilities and attacks? I would venture to say that only by moving at the speed of DevOps—and within the parameters of the DevOps model—can organizations not only meet mandates but also effectively reduce risk for the business.

    Indeed, it’s not a matter of whether DevOps is viable within the current security climate but what requirements need to be added to support and enhance current DevOps work.

    New Guidelines, New Processes, New Tools and New Data

    In response to these new threats and regulations, the software community has been working on developing and updating guidelines to help organizations improve the security of their supply chains—for example, SLSA, NCSC UK supply chain security guidance and NIST cybersecurity supply chain risk management. However, these new guidelines can be complicated and require new processes and new tools.

    Before adding any new tooling or changing any processes, it’s critical to examine existing tools to determine whether and to what extent they satisfy supply chain security mandates. Organizations must also evaluate how well existing processes work to protect the business, then strategically add/subtract from there as needed.

    No matter what solutions are leveraged, more and different tools generate reams of more and different data. What’s important — and to whom? How do I manage the data? When can I trust it? Where do I store it? What problems does the new data help me solve? Organizations will need a way to effectively sift this information and deliver the right data to the right teams at the right time. To preserve the ability to quickly and continuously innovate, it will be important to focus on shifting security left as well as integrating automation whenever and wherever possible.

    As new security metadata becomes available, such as from SBOMs, new solutions for managing that metadata will be key. An open source initiative sponsored by Google, GUAC is designed to integrate software security information, including SBOMs, attestations and vulnerability data. Users can query the resulting GUAC graph to help answer key security concerns, including proactive, preventive and reactive concerns.

    Continue to Shift Security Left

    Another way that organizations can maintain their DevOps momentum while protecting the software supply chain is to continue to invest in shifting security left in the software supply chain. The software community has been talking about this for some time, but many organizations struggle to implement impactful changes. Security teams must work to understand how software is built and collaborate with development and DevOps teams to ensure that improvements are made to both application and infrastructure-as-code supply chains.

    We need to go beyond shift-left vulnerability analysis to ensure that we are evaluating the quality of configuration-as-code (for both applications and infrastructure), that we are taking advantage of new recommendations, and that we are leveraging new attestation solutions designed to work with pipelines such as sigstore and Tekton Chains. After all, the sooner an issue is discovered, the more quickly it can be addressed and the less disruptive and costly it is to fix it.

    No One-Size-Fits-All Solution

    While all of these solutions mentioned here are helpful, there is no single product or framework that will enable organizations to maintain the security-innovation balance. Success, rather, will require a focus on agility, flexibility and effective communication and collaboration—the very hallmarks of the DevOps model.

    Rather than being obviated by the demands of software supply chain security, DevOps provides the opportunity to integrate the kinds of protections that can prevent or mitigate supply chain vulnerabilities and breaches. DevOps also enables organizations to quickly and effectively respond to new threats and new requirements.

    With that said, there’s no doubt that organizations will need to think differently about how they manage software security, including the strategic evaluation and integration of tools that will meet current and future regulatory mandates and, more importantly, protect the business over time. Organizations may also need to think about weighing security over speed, at least until all of the pieces of the software supply chain protection puzzle are in place—but also when future circumstances warrant a fresh look at how to best balance the DevOps pace-protection balance.

  • 6 Transformative Mainframe Predictions for 2024

    6 Transformative Mainframe Predictions for 2024

    The success of organizations in today’s digital economy demands speed — the ability to quickly respond to market trends with new applications and services, instant access to critical data, and resolution of issues even before they affect service. As we move into the new year, the adoption and adaptation of emerging technologies on the mainframe will enable enterprises to embrace changing market forces, turning them into a competitive advantage.

    Below are six predictions of how mainframe organizations will leverage new technologies to advance the platform in 2024.

    Prediction: AI/ML Adoption Grows

    The roles of artificial intelligence (AI) and machine learning (ML) and their adoption across the mainframe, from application development to monitoring to data management, will continue to grow.

    The generative AI revolution is real and here to stay. In the coming year, organizations will continue to explore best-use cases for its integration into mainframe management. Development teams will leverage key AI-recommended insights into automated workflows and code generation, improving developer productivity. Database administrators (DBAs) will look to automate data management processes to efficiently handle the demand for the diverse datasets required by generative AI tools, and mainframe operations teams will increase their reliance on AI-driven predictive analytics and root cause analysis.

    As adoption increases and experience is gained, organizations will work past their AI trust issues and begin to harness the power of automation to become a fully autonomous digital enterprise. At the same time, there will be a growing need to govern AIOps and MLOps to ensure the validity and accuracy of ongoing model development.

    Prediction: DevEx Comes to the Forefront of Companies’ Innovation Strategies

    With the long-awaited shift of mainframe workforces from those with decades of experience to a new generation of stewards now in full effect, organizations will find a new imperative to attract and retain top talent by providing developers with familiar tools and development environments.

    Professionals who have recently graduated from college or are moving from other platforms bring with them an expectation of integrated and automated DevOps practices. The move towards enterprise-wide DevOps and continuous integration and continuous deployment (CI/CD) pipelines will continue, with many companies also looking to employ cross-platform tooling. By giving developers the freedom to use their integrated development environment (IDE) of choice, organizations can decrease onboarding time and get new mainframe developers up to speed faster, enabling them to understand code and its function quickly and easily, and creating a seamless, frictionless way to make changes, debug, and deploy.

    Prediction: Database DevOps Bridges the Gap Between DBAs and Developers

    Changing customer demand and the need for agility will accelerate the drive for developers to seamlessly access and manage data sources. The demand for an agile development environment that extends from applications to databases will increase the inclusion of database DevOps in CI/CD pipelines, including version-controlled database schemas and data, automated database provisioning, and database testing as part of the pipeline.

    By embracing agile practices, such as incremental database changes and rolling deployments, organizations will bridge the gap between DBAs and developers and ensure that mainframe databases evolve alongside application code. This cohesive approach will enhance collaboration, reduce conflicts, and expedite application development, positioning mainframes as agile and adaptable components within the rapidly changing IT landscape.

    Prediction: Democratizing Mainframe Data

    The move ― and the need ― to democratize mainframe data, making it available to AI/ML analytics engines, will gain momentum.

    Organizations have matured their data lakes and gained valuable insights from data housed on distributed and cloud platforms, but the efforts needed to extract and transform mainframe data stored on tape and virtual tape libraries (VTL) has left volumes of business-critical data behind. By operationalizing mainframe integration with the cloud, organizations can make it easier and faster to access mainframe data and transform it into open formats, making it accessible to analytics engines and AI/ML applications.

    The ability to realize the hidden potential of data they already store will make cloud-based storage of mainframe data a prerequisite for enterprises looking to digitally transform their businesses and gain competitive advantage.

    Prediction: Object Storage Becomes Critical for Security

    Mainframe-dependent organizations will seek to integrate hybrid cloud object storage to fortify mainframe data protection against escalating cyberthreats and ransomware attacks.

    In 2024, cyberthreats will almost certainly increase in number and sophistication. The European Union’s Digital Operational Resilience Act (DORA) seeks to ensure that organizations remain vigilant and protect their systems and their customers’ data against increasing threats.

    With DORA set to come into effect in January 2025, and further regulations and guidelines on the horizon, managing mainframe data across diverse locations while prioritizing immutable data copies will be imperative. Organizations will pivot towards proactive strategies, aligning their mainframe infrastructure with hybrid cloud solutions to meet evolving compliance standards and shield against the growing sophistication of cyberthreats.

    Prediction: Innovation Takes the Lead

    Organizations will adopt ruthless prioritization, shifting focus from maintaining services and applications to building new ones for the future.

    Having seen firsthand the success of ongoing mainframe modernization efforts, organizations will place a new focus on building skills and specialties that align with future goals and product roadmaps, putting them in a position to capitalize on coming trends and giving them the ability to quickly respond to changing market forces with innovative new products and services.

    As companies mature their approaches to AI on the mainframe, many will turn to building “applied AI” teams tasked with developing proactive solutions to better analyze data and forecast trends. In short, organizations will think logically about how to embrace AI rather than being embraced by it.

    In 2024, mainframe organizations will accelerate the adoption of emerging technologies like AI and the cloud to increase the adaptability, resilience, and efficiency of the platform while situating it for even greater success in the future. The mainframe will continue to be a platform of innovation, integrating with new technologies and practices borrowed from other platforms to further the evolution of a hybrid IT environment that meets and exceeds the demands of the modern digital economy.

    The role of AI, the impact of data governance regulations, the mainframe skills gap, affordability, and interoperability requirements are all brewing a perfect storm around the mainframe that will mark 2024 as a year of significant transformation that will define the leaders from the laggards.

  • Software Development in the Age of AI: How to Balance Quality and Speed

    Software Development in the Age of AI: How to Balance Quality and Speed

    There’s no mistaking the reality of AI’s influence. Across industries, businesses are calling on tech workers to make the most of this transformative technology as demand skyrockets and companies race to overcome the competition. A flagging economy only pushes expectations for AI use—in software especially. To contend with arduous workloads, 92% of developers are already using AI code generators, according to GitHub. That’s an overwhelming majority, not to mention that this usage is poised to continue generating steam.

    In light of this, we must ask ourselves a critical question: How can businesses ensure developers retain the quality of their AI-generated code? AI is a useful complement to human developers to maximize their time and allow them to focus on projects best suited to their skill sets. That said, companies have to put safeguards in place to harness the technology, not just for boosting developer productivity but also for quality outputs.

    AI Can’t Replace Human Developers

    AI code generators are a useful tool for a developer’s toolbox. Efficient, easy to use, and helpful (in a fairly rudimentary way in these early AI days), they give developers the ability to outsource some code-writing, which helps alleviate mounting demands as companies expect them to produce massive amounts of code — and fast. As most developers continue to buy into AI for productivity benefits, though, comes the responsibility to gut-check what these AI tools produce.

    There are multiple limitations and risk considerations when it comes to AI-generating code. Lack of source knowledge and context is one of them. AI pulls from what it’s taught (i.e., past data, natural language processing, machine learning), meaning it combines too many sources to trace specific pieces of coding back to one place; it’s impossible to trace the code back to a single developer or project. Security, too, becomes an issue; you can’t guarantee that what is produced is safe or clean as is. And, finally, AI doesn’t check for quality, which increases the likelihood that the code will be difficult and costly to maintain. All this ladders up to the mounting technical debt companies are already contending with.

    Developers can’t assume that any AI-generated code is of quality to be put directly into production. Human workers must review code thoroughly to ensure it meets clean code attributes–secure, reliable and maintainable. Ignoring these tenets puts businesses at risk of falling victim to the trillion-dollar cost of bad software. That said, there are still undeniable benefits to using AI in a coding environment that make it an invaluable tool for any company that wants to stay at the top of their software game.

    Working Responsibly With AI in Software Development

    Despite its potential shortcomings, when used safely and effectively alongside a clean code solution, AI brings a tremendous opportunity to the software space that has already made it a key tool for developers as their workloads and demands mount. For one, AI has the incredible ability to churn out code at a speed that far outpaces its human counterparts. This tech can complete tedious work that may take valuable time out of a developer’s day in a matter of seconds. AI can not only move a project forward at a faster rate, but its quick coding capabilities can also give developers a jump start on their projects and help them figure out where to begin.

    While the source of AI’s knowledge base can present a challenge, the vastness of that knowledge is a benefit. As long as a developer can input their preferred language and their specific needs, AI can share the best approach to solving a problem. Its algorithmic assistance can help lead a developer through parts of their projects step by step. This also alleviates the burden of research from developers, who can spend less time checking for accuracy than they would trying to understand different programming concepts or code snippets outside their area of expertise. AI can also handle the slog of documentation, which can not only bring clarity to what didn’t work during the development process but reduce time to delivery overall.

    Developers who don’t harness AI do so at the risk of falling behind the competition as this tech becomes such a vital piece of everyday work in software. But there’s no mistaking the fact that to best incorporate AI, developers still need to ensure its accuracy before any code generated by AI makes it to production.

    GenAI Requires Automated Tools as Safeguards

    As I previously said, developers can’t assume that any AI-generated code is clean and fit for production. That means human skills are still required to check the output. However, this kind of work can often fall to developers or DevOps, and having human workers check code on their own creates an incredible workload that can even negate the use of AI in the first place.

    The best way for companies to mitigate this and mitigate technical debt is to implement automation in the part of the process that scans and monitors AI-generated code. But that can’t be done without developer buy-in. Those creating the software are the people who will best understand the development process as they work through it day in and day out, so their input is required to make the most out of automation.

    There are developer tools that can handle this kind of work while still including developers in the process. For example, introducing code-scanning into the CI/CD process can not only help continuously monitor AI-generated code for bugs and issues, allowing developers to focus on other more challenging aspects of their projects, but it also gives them insight into errors and allows for speedier resolutions. These tools aren’t necessarily new to development, but their ability to identify bugs can help developers prioritize other aspects of the process, particularly as the amount of code reaches exponential levels compared to decades ago with the introduction of GenAI.

    Static application security testing (SAST) also allows for the identification of security vulnerabilities that can potentially arise from AI-generated code. This is critical in the software development lifecycle (SDLC) so developers can spot vulnerabilities before the code goes into production. AI-generated code is only as strong or secure as that source, and since it’s not entirely possible to verify, developers need to take every precaution available to avoid security issues that rack up costs in the long term.

    The vital part of using these tools is ensuring they’re integrated into the process in a way that makes sense for developers rather than creating additional work and headaches with a cumbersome process that doesn’t fit the way teams think and work. But, the need to check AI-generated code shouldn’t be underestimated. Developers can’t simply trust what they’re given. By including a level of automation within the development process—and doing so early and often—developers are in a position to produce better-quality, more secure software to push a business forward.

    Effective Adoption of AI Will Encourage Business Growth

    While it’s difficult to predict exactly where AI will take us in the years to come, it’s equally undeniable that it will play a critical part in the future of software development. Yes, AI is already helping to streamline processes and letting developers prioritize the tasks that best take advantage of their skills. But, as noted above, that doesn’t mean the code output is entirely safe, secure or accurate. Developers still need a level of automation to deliver code that best takes advantage of AI while keeping their workload manageable and their focus on other parts of the development process.

    With the right guardrails in place, developers can guide AI to create the kind of clean, secure, and reliable code that aids businesses rather than create costly problems for them, both now and down the line. Human skills, with a boost from AI, plus code scanning safeguards, give organizations the winning combination to put themselves in the best possible place to succeed in a rapid, fierce competitive landscape of digital business.

  • The Practicalities of Open Sourcing

    The Practicalities of Open Sourcing

    I’ve been deeply immersed in the world of developer products for the past decade, and let me tell you, I’ve been quite the open source enthusiast. Over the years, I’ve had the pleasure (and occasional pain) of shepherding open source projects of all shapes and sizes. It’s almost become a rite of passage in some teams – the moment when your APIs and libraries step into the spotlight. After all, customers want to know what kind of software is going into their code concoctions. But, let’s be real, I’ve also witnessed open source used as a bit of a smoke-and-mirrors exercise. So, if you’re contemplating the daring act of open sourcing your projects, here are some things to know before you set out.

    Manage Your Expectations of External Contributions

    If I had a penny for every open source manager disappointed by unrealistically high hopes of a flood of external contributions, I’d have enough pennies to… well, still not meet those expectations. The truth is that the majority of open source projects never witness a bustling community of external code contributors. Just because your software is popular doesn’t guarantee open source success. Open source is like nurturing a garden; it takes time and effort to grow. Building a community involves shaping your APIs, fine-tuning functionality, defining boundaries and managing expectations. Expecting an avalanche of external contributions in the early days is a recipe for disappointment.

    OSS can be Intimidating

    While developers generally relish the idea of sharing their work with the world, I’ve stumbled upon situations where junior engineers were a tad shy about coding in the public eye. In some cases, we’ve had to delay open sourcing because engineers wanted to tidy up their work before unveiling it to the world. Especially if your team is a mix of early-career engineers, it might be easier for senior team members to offer coaching and conduct code reviews in a more private setting without the intimidating public glare.

    Issue Triage and the Support Price Tag

    In theory, maintaining a public issue tracker is distinct from open sourcing the code itself. In practice, these two are closely linked. Opening up your product’s priorities to the public means putting them on display alongside real interactions and feedback from customers. This doubles the importance of being responsive to customer issues, efficiently triaging them, and setting the right external expectations. For successful open source projects, this is a genuine labor of love.

    The Need for a Private Mirror

    There comes a time when you need to work on something in private, perhaps in preparation for a grand launch event. Unfortunately, last time I checked, GitHub doesn’t offer private branches. You might find yourself wrestling with the challenge of setting up a private mirror. What happens when you’re juggling a significant launch on an internal mirror while simultaneously managing customer bug fixes on the open source repository? Mirrors are like trying to keep two squirrels synchronized – challenging.

    Saying Goodbye to Obscurity

    Some software on the internet relies on obscurity for its security. Open sourcing it would be like sending hackers an engraved invitation to a party they don’t need directions to.

    Unfriendly Dependencies

    If your project is tangled up with third-party libraries sporting licenses that are less open source-friendly than a cactus in a desert, you’re in for a headache. I sincerely hope you don’t run into this, as unfriendly licenses often come hand in hand with unfriendly people.

    Navigating the Ecosystem: Dealing With Dependency Upgrades

    Open source codebases often face a common challenge – the relentless pressure to keep dependencies up to date. This is especially true for libraries and APIs embedded in customer applications, where managing transitive dependencies and supporting older versions can be a delicate balancing act.

    Teams typically adopt one of two approaches. Some isolate these dependencies in a separate namespace to avoid conflicts, while others enforce a policy to always use the latest versions. Regardless of the chosen method, open source projects frequently encounter external contributions aimed at upgrading dependencies. Handling such contributions publicly, and sometimes declining them, can be a tricky task.

    Maintainers must navigate this terrain diplomatically, explaining that the proposed change might not be suitable for all users by paraphrasing, “You’re not our only customer, and this could affect many others.” This balancing act showcases the challenges of maintaining open source projects in a dynamic ecosystem.

    Unexpected Benefits

    Now, let’s not forget the unexpected perks I’ve stumbled upon during my open source adventures. After all, I’ve done enough complaining.

    ● Modernization: Open sourcing often nudges you to modernize your build and deployment pipelines because, frankly, it’s easier to get them right than to get them wrong.

    ● Privacy and security: Open sourcing can make addressing questions about privacy and security a breeze. In an era of regulatory crescendo, it’s your golden ticket to showcasing your data-handling ballet. It’s like saying, “Hey, world, check out our moves!”

    ● Building trust with customers: I’ve found some outstanding team members through successful open source projects.

    ● Trust in open standards and code: Organizations are increasingly placing their trust in open source software. Your larger customers might care about this more than you’d imagine.

    And that’s a wrap, folks. Best of luck with your open source journey!

  • Five Great DevOps Job Opportunities

    Five Great DevOps Job Opportunities

    devopsy.kinsta.cloud is now providing a weekly DevOps jobs report through which opportunities for DevOps professionals will be highlighted to better serve our audience.

    Our goal in these challenging economic times is to make it easier for DevOps professionals to advance their careers.

    Of course, the pool of available DevOps talent is still relatively constrained, so when one DevOps professional takes on a new role it tends to create an opportunity for others.

    The five job postings shared this week are selected based on the company looking to hire, the vertical industry segment and, naturally, the pay scale being offered.

    We’re also committed to providing additional insights into the state of the DevOps job market. In the meantime, for your consideration:

    LinkedIn

    Hanwha Vision America
    Santa Clara, California
    Senior DevOps Engineer
    $160,000 to $200,000

    Indeed.com

    Tecton
    Remote
    DevOps Engineer
    $156,000 to $231,000

    CareerBuilder.com

    The Pokémon Company International
    Redmond, Washington
    Senior Technical Program Manager – DevOps
    $151,000 to $189,000

    SimplyHired.com

    JP Morgan Chase
    Jersey City, New Jersey
    Lead Software Engineer – FinOps/DevOps
    $147,200 to $215,000

    Dice.com

    XTIVIA
    Remote
    Senior DevOps Engineer
    $120,000 to $140,000

  • Scrimmage for Control: On-Premises Vs. SaaS in the Observability Super Bowl of 2024

    Scrimmage for Control: On-Premises Vs. SaaS in the Observability Super Bowl of 2024

    Observability means data. It encompasses a set of practices and tools designed to track the performance and health of software applications, providing insights into the “why” behind system behaviors, not just the “what.” At its core are data pillars we’ve learned how to work with and hopefully get the most out of—metrics (quantitative data on system performance), logs (recorded events within the system) and traces (detailed views of request paths through the system) that offer a comprehensive view of the system’s operation in real-time. SaaS is where the industry has been turning towards for the past decade. It’s easy to operate and upgrade and takes away maintenance and manpower requirements.

    But, using a SaaS solution where data is stored on the vendor’s infrastructure can be compared to playing a football game at your opponent’s home field. Your team does not own the field and must play along with conditions that are outside their comfort zone. While your team can still strategize, execute plays and score points, the control over the playing environment, including the surface, lighting and even some rules, lies with the owner of the stadium (in this case, the SaaS vendor).

    You don’t get the home-field advantage and the comfort of your familiar surroundings, and you also can’t control the audience’s shouts and cheers in the stands across the arena.

    Just as a visiting team must trust the host’s management of the stadium and adapt to its conditions, organizations using SaaS solutions entrust their operational data to third-party vendors. They must trust its data protection policies and privacy agreements to ensure that the organization’s data is handled securely and in compliance with relevant regulations.

    And even more importantly, they have to adhere to its hosting costs.

    Let’s Leverage the Home-Field Advantage

    The alternative is clear: While playing on your own field, you, the coach—hell, the entire team—know this field by heart, and you know best how to keep and enforce your own rules.

    Just as a football team controls the ball and decides on the play, having on-premises data means a company or organization keeps its data within its infrastructure, offering direct control over its storage, management and security. The organization decides how to deploy, use and protect this data and even how to manage the tradeoff of data costs.

    The downsides? Playing on your home field at all times means you’re running slower than the world out there. You’re not quick to update your software to the newest common practices and you’re tasked with maintaining another stack—your observability stack—on top of building your product, which serves your users.

    There’s Another Way. Play the Data on the Scrimmage Line

    In football, the scrimmage line is the imaginary line across the field where the ball is placed at the start of each play. This line serves as the point of engagement between the two opposing teams, with specific rules governing how players can position themselves and how the play begins.

    What if we can keep the data on-premise, in the home field, a little like having the ball on our side of the scrimmage line, within our team’s possession and control? We have the ball, and we decide and strategize how to advance the ball or score from the position behind the scrimmage line, but we can still interact with the outside world (or the opposing team) under a set of rules to which we both adhere.

    This is exactly where observability solutions are heading: A combination of the advantages of SaaS—easy to update and maintain value, with an easy way to distribute the solution inside the organization, alongside the advantages of on-premise—data privacy, data control and full transparent and controllable data costs.

    I call this approach inCloud. It is a modern on-premises approach—data in your cloud environment, protected and controlled by the user, but still governed, managed and served with a guaranteed SLA just like the best SaaS offerings out there.

    Just like the scrimmage line in football that provides a clear boundary and set of rules for how the game is played, inCloud acts in a similar way. Data is kept on-premises, allowing the organization full control over retention, privacy and data integrations. And on the other hand, inCloud defines a set of strict rules under which a control plane is allowed to operate and pass the scrimmage line, or past the organization’s cloud borderline, to manage and access the data like a SaaS solution does best.

    So when we sit down to watch the Super Bowl this Sunday—let’s use that halftime show to think about where we’ll be taking our organization’s observability capabilities in 2024. Will the data be in our hands, crossing that invisible scrimmage line? Or will we lose to our opponent’s home-field advantage?

    May the best team win.

  • API Management: A Weak Link in the Cloud-Native Chain

    API Management: A Weak Link in the Cloud-Native Chain

    Managing APIs is like navigating unfamiliar terrain during a road trip: There’s an old way and a new way to do it, and the new way is just better in many cases. For API management, the old way of doing things involves practices like writing proprietary scripts, manual transaction monitoring and relying on individual employees to take charge of managing specific APIs – leading to more manual work, slower release processes, slower incident recovery and lower reliability. It’s akin to navigating a highway using a paper map, stopping at gas stations to ask for directions and crossing your fingers that you end up at your intended destination.

    The new way of managing APIs looks like this: Using declarative configurations, automated processes and stateless APIs, teams scale and centralize the tasks of configuring, monitoring and updating APIs. They also deploy API changes several times a day, and they can instantly mitigate incidents in production.

    It’s like navigating with a GPS: A simpler, faster and more flexible approach that gets you where you want to go with much less hassle and risk.

    To prove the point, allow me to walk through the reasons why traditional API management strategies fall short for modern environments, then discuss what it takes to manage APIs better.

    What is API Management?

    Let me begin by defining what I mean by API management.

    I’m referring to all of the practices, processes and tools that businesses use to support the APIs their workloads depend on. API management encompasses API design, development, monitoring, testing and security, as well as making updates to APIs after they are in production.

    These tasks are important, of course, because APIs are everywhere today. They handle 83% of internet requests, according to Akamai, which means that keeping APIs documented, updated and monitored is a critical requirement for virtually any organization that deploys Internet-connected applications.

    Without an efficient and scalable means of managing APIs, it becomes difficult not just to defend against challenges like security risks involving APIs but also to guarantee a positive developer experience. The more time and toil your developers have to invest in API management, the less time they have to do the things they want to do and that matter most to the business – like developing cool apps and bringing them to market.

    The Pitfalls of Traditional API Management

    APIs are not new, and most teams that support them have long had API management practices in place. However, in many cases, those practices were conceived in the era when monolithic application architectures and bare-metal servers or virtual machines dominated the IT landscape. They weren’t designed for today’s cloud-native, software-defined, distributed-everything world.

    As a result, traditional API management falls short in modern environments in several key respects:

    ● Slow change management: When you are continuously pushing out updates to apps and APIs, you need an API management strategy that can handle continuous change. Traditional approaches can’t do this. They are too human-intensive to keep pace with rapid change.
    ● Lack of reliability: If you manage API configurations manually, you face a higher risk of introducing changes that accidentally break something. And if that happens, fixing the issue is slow because you have to reconfigure everything by hand.
    ● Slow recovery speed: When you experience an incident related to an API – such as a latency increase due to a misconfigured API – recovery is slower because it’s difficult to trace the surface-level problem back to its root cause, given the lack of a central source of truth for tracking the API’s status and revisions.
    ● Lack of scalability: Traditional API management uses imperative configurations that are stored in stateful databases, which have to be provisioned for each API you want to deploy. That works if you have just a handful of APIs to manage, but it’s impossible to scale if you need to scale to support dozens or hundreds. It also creates particular challenges if you have geo-distributed teams and infrastructure.
    ● Lack of modularity: Conventional API management architectures are not modular or composable, making it difficult to reuse configuration components between APIs. Instead, engineers are forced to “reinvent the wheel” every time they need to configure a new API or move an existing one from dev/test to production.
    ● Lack of standardization: Traditional APIs typically depend on proprietary technologies. As a result, they’re difficult to integrate with modern stacks that are based on widely used tools like Kubernetes, OpenTelemetry and Git.
    ● Poor collaboration: Documenting APIs and sharing monitoring insights between teams is also challenging when configurations are imperative and there is no single source of truth that all stakeholders can refer to when tracking the status of an API.

    These challenges lead to a nightmare for teams tasked with Day 2 operations, meaning those that support APIs after they are in production. Slow, inefficient and unscalable management processes lead to struggles in detecting and fixing problems with API performance or security, as well as in updating APIs. And again, they hamper the developer experience by forcing developers to devote time and energy to deciphering APIs and versions to use, finding up-to-date documentation, and so on. As a result, developers whose main job is to write software, not manage APIs, become burdened by API management requirements.

    A Better Way to Manage APIs

    The good news is that a better world is possible – at least when it comes to API management. For modern apps and environments, API management should be rooted in the following principles:

    ● Composability: Your API management stack should be composable so that you can pick and choose from different ingress controllers, monitoring and analytics tools, load-testing solutions and so on. You shouldn’t be bound to a rigid, inflexible “stack.”
    ● Statelessness: Stateless API management, which frees APIs from dependency on stored configurations, enables scalability, simple rollbacks and consistency across environments because you can deploy APIs anywhere with minimal provisioning.
    ● Automation: The more you automate the processes involved in API management, the faster you can implement changes and the lower your risk of mistakes and inconsistencies due to manual management.
    ● Immutability: The underlying infrastructure that powers your API management stack should be immutable so you can replicate it easily and quickly – a benefit during processes like troubleshooting following a rollback.
    ● Auditability: You should be able to maintain a timestamped log of every change to your API that documents when changes are made, who made them, why they happened and what the potential implications are.
    ● Open and standards-based: The ability to monitor and manage APIs using open, standardized tools and frameworks – such as OpenTelemetry and kubectl – also leads to consistency and simplifies collaboration between teams. In addition, API contracts should be standardized using specifications like OAS.
    ● Efficient incident response: When you discover a performance problem or failure linked to an API, you should be able to troubleshoot and resolve the issue quickly without having to chase down documentation or find the one person at your company who understands how the API works.

    When you manage APIs according to these principles, you escape the tedium and inefficiency of traditional API management. By extension, you can easily support the many APIs that tie your cloud-native environments together.

    The Role of GitOps

    Going a step further, a declarative, stateless, automated approach to API management also enables GitOps, meaning that you can manage APIs via declarative configurations that you store in Git repositories.

    This practice also leads to greater reliability (thanks to consistent, repeatable and scalable configurations), collaboration (because Git serves as a shared, central source of truth for API configurations and workflows) and efficiency (due to the ability to automate and customize deployments using Git-based triggers and actions).

    Conclusion: API Management at Cloud-Native Speed

    In a world where more than half of existing applications are cloud-native and a majority of new apps will leverage cloud-native principles, traditional API management just doesn’t work for these new platforms. It leads to change management processes that are slow and unreliable – exactly the opposite of what businesses need to thrive in a dynamic, cloud-native world. It also slows down recovery, leading to lower levels of reliability and higher risk for the organization. On top of this, it’s tedious, inefficient and unscalable. So yes, cloud-native API management is currently broken.

    We need a radical new approach to API management with one goal in mind: Automate Day 2 API operations with a modern approach. Modern solutions should be rooted in stateless, declarative configurations, open standards and Git-based automation. They must enable change management with high reliability and instant recovery, so you don’t have to worry about API management becoming the weakest link in your ability to innovate quickly and with minimal risk.

    We envision a new world where managing APIs is no longer a burden that developers and platform teams must bear but instead a process that helps teams innovate as quickly as possible, no matter how complex the API architectures they are working with. We also want to help businesses bring products to market faster, as well as troubleshoot incidents more quickly and with less effort.

    Ultimately, we want to empower organizations to take full advantage of cloud-native technologies and paradigms by adopting a cloud-native approach to API management.