devopsy.kinsta.cloud

Tag: LLM

  • Using LLMs to Automate Pipeline Conversions From Legacy to Tekton

    Using LLMs to Automate Pipeline Conversions From Legacy to Tekton

    There have been many technological advancements in the market today, but the main hindrance is the cumbersome process of switching from the existing technology/framework to the new one. This is where teams and organizations are left behind in the tech curve. In the DevOps landscape, CI/CD pipelines are the backbone of DevOps, and how you work with them largely dictates your DevOps efficiency.

    CI/CD pipelines, too, like other major technologies and frameworks in the past, have seen various evolutions; the latest among them is being made reusable in an attempt to standardize CI/CD. However, here too, there are several cons:

    1. The pipelines are not proprietary to end-user enterprises and belong to the DevOps platforms which are not open-source
    2. Migrating to these platforms and pipelines is a huge task
    3. They may not be as efficient as depicted

    Speaking of new advancements in technology, most might be familiar with Tekton for CI/CD pipelines: It is Google’s open source framework that is now a graduated CDF project. It redefines how pipelines are built (through reusable tasks) and these tasks are open source and community-vetted. They allow for pipeline authorship, and enterprises can re-use them across platforms that are compatible with Tekton, thus preventing vendor lock-in.

    While this mitigates the first challenge of pipeline ownership and migrations, however, the same problem plagues DevOps teams – transitioning from existing legacy technologies/frameworks to newer, much more capable frameworks like Tekton.

    However, what if there was a way to seamlessly automate your transition from legacy pipelines to cutting-edge, reusable Tekton pipelines without toil? This is precisely what the article aims to explore – automating the conversion of Yaml scripts of existing pipelines to Tekton Yaml scripts.

    Let us consider the three most popular CI/CD platforms for this use case – Jenkins, GitLab and Azure DevOps. Before we proceed to the proposed approach to achieve this (which is detailed in the second half of this article), let us first look at the limitations of Jenkins, GitLab and Azure Pipelines and why one should convert.

    Common Limitations of Jenkins, Azure and GitLab Pipelines

    Resource Intensive: Legacy pipelines can be resource-intensive, especially for larger organizations or projects. Managing server resources, optimizing configurations and ensuring sufficient hardware can be a challenge.
    Complex Setup: Setting up and configuring Jenkins, GitLab and Azure DevOps can be complex, particularly for self-hosted instances. This complexity may require experienced administrators and substantial time investment.
    Plugin and Integration Compatibility: All three platforms may face compatibility issues with certain plugins or integrations.
    Scaling Challenges: Scaling in organizations, especially when dealing with multiple microservices, necessitates a shift from individualized scaling to a template-based approach. Failing to do so leads to a chaotic increase of pipelines, undermining standards and organizational processes.
    Costs: While both Jenkins and GitLab offer free and open-source versions, some advanced features or capabilities may only be available in higher-tier paid plans. This can lead to cost considerations at scale.
    Customization Complexity: These platforms can be complex to customize workflows and integrate custom scripts as they require advanced scripting knowledge.
    User Interface Complexity: Though they have feature-rich interfaces, it can sometimes be overwhelming for new users to navigate the platforms and find specific settings or options.
    Limited Native Integration & Plugins: The native integrations of Jenkins, GitLab and Azure may not cover the full spectrum of tools and services a team may need. Additional integration efforts or third-party tools may be required to bridge gaps in the development workflow.

    As can be seen in the above limitations, many are common including scalability challenges, being resource intensive, limited integrations etc. All the challenges boil down to one common denominator: Complex pipeline management at scale.

    This is where a framework like Tekton can help tackle all the challenges at once, by just easing pipeline management: with its K8s-native pipelines that are reusable, open source and tasks that are community-vetted.

    Tekton Pipelines: Advancing CI/CD with Kubernetes-Native Automation

    Tekton, an open-source project, is rapidly gaining traction as a game-changer in the world of CI/CD pipelines. Its Kubernetes-native design is a standout feature, seamlessly integrating with Kubernetes clusters to harness its scalability and resource efficiency.

    Furthermore, Tekton’s cloud-native roots ensure it can execute tasks in lightweight, ephemeral containers, optimizing resource utilization. The ability to declare CI/CD pipelines as code, coupled with GitOps practices, enhances automation and reproducibility.

    How are Tekton Pipelines Different?

    Kubernetes-Native Automation: Tekton pipelines are deeply integrated with Kubernetes, utilizing Custom Resources and Operators to define and execute CI/CD workflows.

    Declarative and Modular: These pipelines are defined declaratively using YAML manifests, promoting version control, collaboration and code reviews. The modular design of tasks and pipelines enables teams to create reusable components for building, testing and deploying applications.

    Containerization and Isolation: Each task in a Tekton pipeline runs within a dedicated container, ensuring isolation and reproducibility of the build and deployment environment.

    Event-Driven Architecture: Tekton pipelines can be triggered by various events, such as code commits, pull requests, or external events, providing a responsive and event-driven CI/CD process.

    Scalability and Efficiency: Tekton pipelines leverage Kubernetes’ inherent scalability, enabling efficient execution of tasks across a dynamic pool of worker nodes.

    Visibility and Observability: They offer built-in monitoring and logging, allowing teams to gain insights into pipeline execution, track progress and diagnose issues.

    Integration and Extensibility: Tekton pipelines can be easily extended with custom tasks and integrations, enabling teams to connect with various tools, services and cloud platforms.

    Cloud-Native Best Practices: By embracing Tekton pipelines, organizations align with cloud-native best practices, promoting the use of Kubernetes and containerization for streamlined application delivery.

    Why Transition From Jenkins, GitLab, and Azure Pipelines to Tekton Pipelines?

    Migrating from Jenkins, GitLab, or Azure pipelines to Tekton pipelines can be a strategic choice, especially for organizations deeply entrenched in Kubernetes and cloud-native methodologies.

    Optimize Kubernetes-Centric Workflows: Projects tightly integrated with Kubernetes may find Tekton pipelines more aligned with their ecosystem.

    Accelerate Cloud-Native DevOps: Organizations aiming to embrace a cloud-native approach and leverage Kubernetes infrastructure.

    Performance and Scalability: Tekton pipelines run on clusters that can dynamically be created and destroyed according to demand, thus taking resource optimizations to the next level.

    Cloud-Native Best Practices: Best practices, such as containerization, declarative configuration and infrastructure-as-code are a natural fit due to their Kubernetes-native design.

    Proposed Workflow for Automating Pipeline Conversions at Scale

    The overarching objective of this workflow is to facilitate a seamless conversion process using large language models (LLM) without delving into the intricacies of fine-tuning but rather using the benefits of prompt engineering. Central to this approach is the use of the chat context and the strategic application of roles for LLM models to establish context. Along with this, sample pipelines are leveraged to train and fine-tune the model to define how the corresponding YAMLs of all these platforms look for any given pipeline.

    The use of prompts and instructions tailored to fit the use case of generating contextual pipeline YAML and conversion of Jenkins/Gitlab pipelines to Tekton pipelines makes this workflow possible. We maintain a context and a contextual mapping of real-world examples and the most appropriate prompt designs to produce the most accurate responses. The role of users in this context adds weight and further refines the response received.

    However, before proceeding into details and examples, let’s set the base for what exactly generative AI means and its context in our use case:

    Generative AI and LLM for Automating YAML Script Conversions

    Generative AI, such as LLMs, create content like text, benefiting from patterns in large datasets. Large language models (LLMs), with billions of parameters, excel in generating human-like text by understanding language structures. They have broad applications, including natural language understanding, summarization, translation and content generation, promising automation and human-like interactions with AI.

    Adapting the Model to Solve Real-world Use Cases

    Fine-tuning and prompt engineering are two common methods used to train large language models (LLMs) like GPT-3, BERT and others. These methods help adapt pre-trained models for specific tasks or generate contextually relevant responses. Both fine-tuning and prompt engineering play crucial roles. Fine-tuning helps the model comprehend complex use cases, while prompt engineering provides detailed requirements, extracting the best results from the model.

    Why Prompt Engineering?

    Prompt engineering stands out as a highly advantageous approach in specific use cases where precision, control and customization over AI model behavior are paramount. This method empowers developers and users to meticulously design input prompts, effectively guiding AI models to produce contextually relevant and desired responses.

    Prompt engineering is both an AI engineering technique for refining large language models (LLMs) with specific prompts and recommended outputs and the term for the process of refining input to various generative AI services to generate text or images.

    The process of prompt engineering to fit our use case includes :

    Defining the goal: The first step in the process of AI prompt engineering involves setting a clear objective.

    Crafting the initial prompt: With the goal in mind, it’s time to draft an initial prompt. This could take the form of a question, a command or even a scenario, depending on the goal.

    Testing the prompt: The initial prompt is then input into the language model, and the response is analyzed.

    Analyzing the response.

    Refining the prompt: With the insights gathered from testing and analysis, it’s time to revise the prompt. This could involve making it more specific, adding more context or changing the phrasing.

    Iterating the process: The testing, analyzing and refining steps are repeated until we are satisfied that the prompt consistently guides the model toward generating the desired response

    To conclude, here’s an example of typical YAMLs from Jenkins, GitLab and Azure DevOps, and how the same YAML looks when converted to a Tekton YAML. This goes a long way in facilitating teams to finally make a jump from their existing pipelines to cutting-edge reusable pipeline frameworks:

    Jenkins YAML

    GitLab YAML

    Azure YAML

    Converted Tekton YAML

  • Google De-Recruits 100s of Recruiters ¦ ARM Valued at $45½B in IPO

    Google De-Recruits 100s of Recruiters ¦ ARM Valued at $45½B in IPO

    Welcome to The Long View—where we peruse the news of the week and strip it to the essentials. Let’s work out what really matters.

    This week: Google fires hundreds of recruiters, and ARM gets a sky-high valuation. (more…)

  • Steady On. We Still Have a Job to Do

    Steady On. We Still Have a Job to Do

    It is all the rage to talk about how significantly large language models (LLMs) and generative AI will change every niche and corner of IT. And change IT it will. But the hype cycle is at its highest right now, and I’d caution against getting too worked up about it. We’re in for a few interesting years, and the spaces I cover (DevOps and security—with a more application-centric perspective but with ops experience) are definitely going to benefit from the various AI bits being worked in. All testing—including AppSec, but all testing—will benefit both in test generation and results analysis/filtering. In fact, I’ll go ahead and say that the question in testing will no longer be, “How much time to we have to devote to it?” and will rapidly become “How many systems’ resources do we have to dedicate to it?” It will still be a long process—AI isn’t speeding up the cumulative time it takes to run thorough testing (be it functional or security). It will make that the bottleneck though, because test generation will at least be LLM-assisted, and will eventually become completely generative AI-created while at the same time, results will get more and more filtering and analysis. A minor result here plus a minor result there will automatically be detected as an extended vulnerability. There are a ton of other possibilities, but these two seem pretty obvious when I look across the testing technology space. More testing and better results analysis are on the horizon—without requiring more staff. Of course, you can do so much more, which would require the organization to add staff to follow up on results, but that’s nothing new; testing finds issues that are interruptive and take man-hours to fix—we all know of bits in the application portfolio that we should probably fix but that aren’t high enough priority to do so, for example. But fixing is different than finding. I have seen demos in security scanners that offer auto-fix options for developers. Run the test during CI, create a ticket and let the developer see suggested code rework right in the IDE. It is early days for this, but if you look, you can find vendors doing it. And that’s really rather cool.

    But we’ve all heard the “this changes everything” storyline over and over. 4GLs, XML, Java and JavaScript were all going to eliminate developers. In fact, they all created more developers. AI has a better chance of actually removing parts of our workload than any of these technologies did, but we’re still in “watch and wait” mode. It seems obvious that grunt coding will fall to LLMs … And aside from keeping staff trained to manage source, that is a good thing. We use open source, modules and libraries to do much the same thing today. I see this as equivalent to entry-level writing jobs. We’ll have to transform from the current model, but the current entry level development model wasn’t ideal anyway, so lets make a better one. Testing capabilities will be greatly enhanced, but testing of all kinds was so far behind new code generation at most orgs that the impact will be all positive as testing catches up. AI-based test generation today can generate thousands of permutations for a given piece of code, and analyze the results to just have analysts look at the truly disconcerting or confusing test results. And the question will become, “How many servers do we want to dedicate to run this?” Which is different, but just means we have more tests running to make a more solid and/or secure product and environment.

    So, in summary, is change coming? Yes. And that is not significantly different than the last couple decades where change has become IT’s constant companion. For most of you, the way you do your job today is not the same as the way you did it even two years ago. ML/GAI/LLMs are just another iteration. The apps still have to be built, deployed and maintained. So until HAL comes along, keep kicking it. You’re keeping it together, and who knows what the future brings.

  • IBM LLM AI: COBOL to Java ASAP ¦ ARM IPO is GO!

    IBM LLM AI: COBOL to Java ASAP ¦ ARM IPO is GO!

    Welcome to The Long View—where we peruse the news of the week and strip it to the essentials. Let’s work out what really matters.

    This week: Translating legacy COBOL code to a slightly more modern language, and Arm will go public (again) next month. (more…)

  • Using Generative AI for Issue Remediation and Software Reliability

    Using Generative AI for Issue Remediation and Software Reliability

    In today’s fast-paced digital landscape, keeping your services up and performant, especially for cloud-based and online services, is crucial for staying competitive. Downtime or latency issues can drive customers away, especially when all it takes is a click to switch over to a competing SaaS solution. DevOps and site reliability engineering (SRE) teams face the constant challenge of minimizing mean-time-to-remediation (MTTR) when dealing with errors and issues. As valuable as search engines like Google are, the sheer amount of information available can make finding the right solutions a time-consuming task. What if there was a way to automate and streamline this process, making error investigation more intelligent, focused and efficient?

    This article will walk you through the journey of error resolution from the first obscure log line to identifying the problem hidden within it. We’ll resolve the issue using several tactics, ultimately leading to the use of generative AI to cut MTTR and provide precise IT recommendations. This journey includes useful measures and principles that can be applied by DevOps teams and explores the potential for applying generative AI.

    The Current Experience of DevOps/SRE

    Picture this: A log line pops up with an obscure error message, and the first instinct is to search for this message on Google for potential solutions. After all, someone else must have encountered a similar problem, right? However, the abundance of resources and search results can be overwhelming. The search results are often displayed based on website relevance, not necessarily the relevance of the error itself. Consequently, precious time is wasted sifting through countless search results, leading to a longer time to understand and remediate the error. For DevOps and SRE teams responsible for maintaining system stability, reducing MTTR is a crucial key performance indicator (KPI). This begs the question: How can we leverage automation and artificial intelligence to improve the search process and accelerate error resolution?

    Our First Step: Cognitive Insights

    In our organization’s initial attempt to tackle this challenge, we focused on leveraging crowdsourcing techniques to get more relevant results than the brute-force Google search. Moreover, we wanted to automate the process and run it offline so that when an incident occurs, we can offer useful insights rather than starting the search as the system is down or malfunctioning.

    The approach involves an offline phase and an online phase. The offline phase involved analyzing all our ingested logs and identifying common log patterns. This also allowed us to count how often this pattern occurred in our system and how prevalent it is. Then we crawled relevant technology forums like StackOverflow, Google Groups and Bing Groups for relevant discourse around these log patterns that could offer relevant solutions. The next step was to rank the search results by relevance and take the top five most relevant links.

    The offline phase resulted in a library of known log patterns and, for each one, a cognitive insight containing these links, as well as additional information such as the severity level, how many times this pattern occurred in the logs, the date of first occurrence and additional tags of involved technologies, tools and domains.

    The online phase occurs in real-time as new logs come in. Upon ingesting the log, it is automatically matched against all known patterns; if it matches one, it gets the cognitive insight for that pattern. This means that as soon as the problematic log comes in, the DevOps engineer already has focused and ranked search results and context to start from, which accelerates the investigation process.

    The Next Step: Why Don’t we ask Generative AI?

    After reflecting on our initial approach, we had an epiphany. Large language models (LLMs) like ChatGPT had already crawled the web and absorbed a vast amount of information. So, why not leverage their capabilities and ask them directly for insights? The idea was simple: Let the AI “read the posts” for us and provide us with recommendations. We started by formulating specific questions such as “What could the possible errors be?” and even went a step further by asking for further investigation steps. However, implementing this seemingly straightforward approach came with its own set of challenges. We needed to run preprocessing before asking the generative AI, as well as running post-processing on the answers it returned, to get what we expected. Let’s see what that process entails.

    How we Did It: Analyzing, Sanitizing and Validating

    Before asking ChatGPT (or any other generative AI tool) for assistance, several preparatory steps were necessary. First, we had to analyze incoming logs, identify patterns and score them based on severity. This allowed us to prioritize the most critical issues.

    Secondly, we had to carefully design our prompts for the generative AI using prompt engineering, ensuring that we framed our questions in a way that yielded precise and relevant responses. Prompt engineering is the practice of strategically designing effective prompts or instructions for language models to achieve more accurate responses. It involves carefully specifying the input format, context or constraints to guide the model’s generation process and obtain more accurate and desired results. This is true for any use of generative AI, and for our case in particular. Beyond accuracy, prompt engineering helped us tune the requested format and length of the answer to fit the “executive summary” paragraph we popped up for the user. This ensured it wasn’t too lengthy nor too short to provide meaningful insights.

    Additionally, we took great care in sanitizing the queries and removing any sensitive data to maintain privacy and security. This step was especially crucial for those working with public services, highlighting the importance of protecting users’ personally identifiable information (PII), such as names, emails, phone numbers, IP addresses or GPS coordinates.

    With all this preprocessing done, we sent the prompt to the generative AI. It is important to note that public services may not always be available (as some returned a “service is busy” message when working with ChatGPT and other services), so automation should take care of delays and employ retries as needed.

    Finally, as responses were received from the AI, it was essential to validate those. We ran an analysis and filtered out non-relevant answers from different domains (after all, a log is also a product of a tree, and Kafka is also a novelist), ensuring there was no offensive content or misleading information. The semantic integrity of the responses was also carefully evaluated.

    These AI insights have proven to be a powerful troubleshooting tool, and we’ve made it an integral part of our service.

    This journey is not unique. In fact, DevOps teams can employ similar measures for their own telemetry data to serve their own operational workflows. These principles, such as data sanitization and validation, even apply to other types of data and use cases.

  • AI ‘is Getting Worse’ ¦ AI ‘Will Lose India Jobs’ (Probably Isn’t ¦ Probably Won’t)

    AI ‘is Getting Worse’ ¦ AI ‘Will Lose India Jobs’ (Probably Isn’t ¦ Probably Won’t)

    Welcome to The Long View—where we peruse the news of the week and strip it to the essentials. Let’s work out what really matters.

    This week, a conundrum: On the one hand, researchers say ChatGPT is losing the plot; and on the other, we hear outsourced coding jobs in India will be replaced by AI. (more…)

  • Forking RHEL! Oracle and SUSE Join the Fight ¦ Silverman Sues AI Firms

    Forking RHEL! Oracle and SUSE Join the Fight ¦ Silverman Sues AI Firms

    Welcome to The Long View—where we peruse the news of the week and strip it to the essentials. Let’s work out what really matters.

    This week: SUSE announces a Red Hat Enterprise Linux fork after Oracle said a similar thing, and Sarah Silverman says OpenAI and Meta have stolen her words. (more…)