devopsy.kinsta.cloud

Category: DevOps Practice

  • Is it Time to Reconsider CI?

    Is it Time to Reconsider CI?

    DevOps is old, in technology terms, at this point. Many of you have never developed under a waterfall and have no idea what tools like Visual Test were, and that’s actually very cool. While knowing other processes and tools does help make informed decisions, being immersed in the tools that matter today is more important. We have entire DevOps stacks built up that allow for the development, building, testing and, increasingly, deployment of apps in an agile environment.

    As always happens, market leaders and those who compete with them, complement them or serve a subset of the market have grown up and become standards. Indeed, application release orchestration (ARO) was a phrase coined in the early days of DevOps that fell away completely; I heard it recently and thought, “Oh, are we bringing that name back?” Vendors have come and gone; some merged into other products, and some are just plain gone. Again, that is a good thing as the market, its needs and the customer base are all mature. While it is terribly painful to be the user of a product that just goes away one day, the prevalence of open source in the CI/CD landscape does buffer that pain a bit. And the market gets better as tools fall away and new ones take their place.

    But sometimes, the movement is slow, just simply due to inertia. Most of us – and by us, I mean IT shops, and by most I mean a lot – use Jenkins. The number seems to be on a steady downward trend, and the tool’s maintainers have definitely taken steps to improve the parts that make organizations consider other solutions, but the trend is there.

    And I, personally, think that is a good thing. Stagnation is what made waterfall such a weakness in enterprise IT. Everyone did it, so that makes it good. Same is true for the old phrases we’ve all heard at least one variant of – “No one ever got fired for buying IBM”, or “No one ever got fired for buying Oracle”, “…Cisco”, etc. The obvious answer to those statements then, as now, was, “Well, maybe they should have.”

    The same is true for Jenkins. It is a great system that can do everything. And that’s part of the problem. If you have small projects or are a team using it standalone with light requirements that all revolve around getting a quality product out the door quickly, Jenkins just might be too much for you. In the early days, Jenkins wasn’t, but the plug-in architecture and years of competition and changing market needs have made it huge. And while there is arguably better documentation than any other CI, and inarguably the largest user base to ask questions of, it’s still weighty to use in place of a SaaS that comes pre-configured and you can just hand it a repository with build files in it.

    There are a lot of enterprises that need what Jenkins has grown into–it did not expand in a vacuum. But there are many, particularly new, small, agile orgs or standalone teams that just don’t.

    There are a lot of options out there, and it is worth taking stock of what you are trying to do with your build/test process, and build a list of things required, then look seriously at what’s available. For some, lightweight is good.

    Of course, as much as one can be in our technological age, I am a minimalist. I like my D&D to be a rules-light variant, my dev tools to have enough functionality but not bury me in extraneous stuff, laugh at Excel for including abstract functions and then selling to consumers, etc.

    At my company that does development, we do indeed use Jenkins. But our projects range from simple knock-it-out one-offs to complex systems with multiple servers and clients, so we feel it is best for that range of variation. And like many others, the investment to move old projects would be questionable without a driving need. Not that I get to do much development these days myself, but I do stay in it just enough to be dangerous. But we have asked the question and determined the answer. For now.

    If you have issues with Jenkins, look around. There are options. They are not for everyone, but your org isn’t everyone. Find the best tool, not the most convenient, and keep kicking rear.

  • Applying AI/ML to Continuous Testing

    Applying AI/ML to Continuous Testing

    Artificial intelligence (AI) and machine learning (ML) can play a transformative role across the software development lifecycle, with a special focus on enhancing continuous testing (CT). CT is especially critical in the context of continuous integration/continuous deployment (CI/CD) pipelines, where the need for speed and efficiency must be balanced with the demands for quality and security. AI/ML contributes by automating complex tasks, predicting potential issues before they occur and providing actionable insights, thereby reducing manual efforts and enabling more strategic use of human resources.
    Moreover, the application of AI/ML extends beyond mere automation of tests. It encompasses the capability to learn from data, adapt to new information, and improve over time. This ability is invaluable for identifying patterns, anticipating vulnerabilities and optimizing testing strategies. In quality assurance, AI-driven tools can predict areas most likely to fail and tailor testing efforts accordingly. In security, ML algorithms can detect anomalies that signify potential threats, while in operations, AI can enhance feedback mechanisms, leading to more resilient and responsive systems.

    AI/ML for Continuous Testing

    Here’s how AI technologies can be applied to reduce bottlenecks associated with testing activities across various testing activities:

    1. Requirements Analysis
    • Explanation: Ensures test scenarios align with business requirements and user needs.
    • Bottleneck: Misinterpretation or incomplete analysis can lead to inadequate test coverage.
    • AI/ML Solution: NLP can automate the extraction and interpretation of requirements, ensuring comprehensive and accurate test coverage.

    2. Test Strategy
    • Explanation: Outlines the testing approach, objectives, and resources.
    • Bottleneck: An unclear strategy may lead to inefficient testing efforts and resource allocation.
    • AI/ML Solution: AI can analyze historical data to suggest the most effective test strategies and predict resource needs.

    3. Test Plans
    • Explanation: Detailed documents guiding the testing process, timelines, and responsibilities.
    • Bottleneck: Inflexible plans can struggle to adapt to project changes, causing delays.
    • AI/ML Solution: Machine learning algorithms can suggest adjustments to test plans based on ongoing project developments and past outcomes.

    4. Test Cases
    • Explanation: Specific conditions under which a test is executed.
    • Bottleneck: Time-consuming development and significant effort to maintain.
    • AI/ML Solution: AI can automate the generation of test cases from requirements documents, improving efficiency and coverage.

    5. Test Scripts
    • Explanation: Automated scripts that execute test cases.
    • Bottleneck: Script development and maintenance can be resource-intensive.
    • AI/ML Solution: AI can generate and update test scripts based on changes in the application or test cases, reducing maintenance effort.

    6. Test Data
    • Explanation: Data sets used during testing to simulate real-world scenarios.
    • Bottleneck: Creating, managing, and maintaining accurate test data is challenging.
    • AI/ML Solution: AI can automate the generation and management of test data, ensuring relevance and variety.

    7. Test Environment
    • Explanation: The setup where testing is conducted mirrors production environments as closely as possible.
    • Bottleneck: Configuration and maintenance of test environments are complex.
    • AI/ML Solution: AI can predict and configure optimal test environments based on test requirements, reducing setup time.

    8. Coordination with Dependent Systems
    • Explanation: Ensuring the system under test interacts correctly with databases and other applications.
    • Bottleneck: Dependency management can cause delays.
    • AI/ML Solution: AI can automate the detection and resolution of integration issues, enhancing coordination efficiency.

    9. Test Environment Setup
    • Explanation: Configuring infrastructure and tools necessary for testing.
    • Bottleneck: Setup complexity and resource contention lead to delays.
    • AI/ML Solution: AI algorithms can optimize environment setup, automatically adjusting resources as needed.

    10. Test Campaign Setup
    • Explanation: Organizing and scheduling a series of test executions.
    • Bottleneck: Requires careful planning and can be hindered by resource limitations.
    • AI/ML Solution: AI can assist in scheduling and prioritizing test campaigns based on risk and impact analysis.

    11. Test Execution
    • Explanation: The process of running test cases and scripts, both automated and manual.
    • Bottleneck: Time-consuming, particularly for manual tests.
    • AI/ML Solution: AI can prioritize test execution and identify flaky tests, streamlining the process.

    12. Test Verdict Reporting
    • Explanation: Determining and reporting the outcome of test executions.
    • Bottleneck: Manual verdict determination can be slow.
    • AI/ML Solution: AI can automatically interpret test outcomes, speeding up reporting.

    13. Logging of Data
    • Explanation: Recording data relevant to the test for further analysis.
    • Bottleneck: Extensive data collection can overwhelm resources.
    • AI/ML Solution: AI can intelligently filter and log pertinent data, reducing noise.

    14. Test Result Analysis
    • Explanation: Analyzing test outcomes to identify defects and issues.
    • Bottleneck: Requires significant time and expertise.
    • AI/ML Solution: ML algorithms can quickly identify patterns and anomalies in test results, highlighting potential issues.

    15. Test Result Reporting
    • Explanation: Communicating findings to stakeholders.
    • Bottleneck: Compiling reports is time-intensive.
    • AI/ML Solution: Automated reporting tools powered by AI can generate insightful and comprehensive reports quickly.

    16. Waiting for Resource to Fix Failed Tests
    • Explanation: Downtime while awaiting fixes for identified issues.
    • Bottleneck: Halts testing progress.
    • AI/ML Solution: AI can predict which areas might fail and propose potential fixes

    Challenges

    Applying AI/ML to software testing activities offers numerous advantages but also introduces several challenges. Addressing these problems requires a combination of technical solutions, process adjustments, and cultural changes.

    1. Lack of Intuitive Understanding of the Application Being Tested
    • Problem: AI/ML models may not fully grasp the application’s context or the nuances of its functionalities, leading to less effective test scenarios.
    • Solution: Enhance AI models with richer contextual data and incorporate feedback loops where testers can refine and adjust AI-generated test cases. Employing techniques like reinforcement learning can also help AI models better understand application contexts over time.

    2. Repeatability and Consistency Between Test Sessions
    • Problem: AI-driven tests may generate different outputs for the same input over different sessions, complicating test consistency and traceability.
    • Solution: Implement versioning for AI models and their training data, ensuring consistency across test sessions. Use deterministic approaches in conjunction with AI to maintain a core of stable, repeatable tests.

    3. Lack of Understanding of the Tests Generated
    • Problem: Testers may find it challenging to understand or trust the rationale behind AI-generated test cases, impacting their ability to evaluate test outcomes effectively.
    • Solution: Incorporate explanations into AI/ML models to provide insights into their decision-making processes. Foster a culture of trust and understanding through education and transparency regarding how AI models operate.

    4. Test Coverage
    • Problem: There’s a risk that AI/ML might not adequately cover all testing scenarios, potentially missing critical defects.
    • Solution: Combine AI/ML with traditional testing methods to ensure comprehensive coverage. Regularly review and adjust the criteria used by AI/ML models to generate test cases, ensuring they align with evolving application features and risks.

    5. Compatibility with Different Test Tools
    • Problem: AI/ML models might not seamlessly integrate with existing testing tools and frameworks, limiting their utility.
    • Solution: Develop or use AI/ML solutions with extensive API support and integration capabilities. Work with tool vendors or contribute to open source projects to enhance compatibility.

    6. Acceptance by Teams
    • Problem: Testers and developers may be skeptical or resistant to AI-driven testing due to concerns about job displacement or mistrust in AI’s effectiveness.
    • Solution: Educate and involve teams in the development and implementation of AI/ML testing strategies. Demonstrate the value of AI/ML in augmenting their roles rather than replacing them, focusing on AI as a tool to tackle mundane tasks and allowing them to focus on more complex and rewarding work.

    7. Data Quality and Availability
    • Problem: AI/ML models require large amounts of high-quality data for training. Inadequate or poor-quality data can lead to ineffective testing.
    • Solution: Invest in data curation and generation strategies such as synthetic data creation to ensure models are well-trained.

    8. Continuous Learning and Adaptation
    • Problem: AI/ML models may become outdated as applications evolve.
    • Solution: Establish continuous learning mechanisms where models are regularly updated with new data and feedback, ensuring they remain relevant and effective.

    9. Ethical and Bias Considerations
    • Problem: AI/ML testing models might inherit or amplify biases present in their training data, leading to unfair or discriminatory outcomes.
    • Solution: Implement ethical guidelines and bias detection methodologies for AI/ML model development and use. Regularly audit models for biases and correct them as needed.

    By addressing these challenges with thoughtful strategies, organizations can maximize the benefits of AI/ML in testing activities while mitigating potential drawbacks, leading to more efficient, effective and trustworthy processes.

    Summary

    This blog explained use cases of artificial intelligence (AI) and machine learning (ML) within the realm of continuous testing of the software development life cycle. By harnessing the power of AI/ML, we’ve seen how organizations can significantly enhance their development processes, making them more efficient, secure and responsive to user needs. The exploration of specific AI/ML applications across various activities has provided a blueprint for automating and optimizing tasks that traditionally required significant manual effort.

  • Leveraging eBPF for DevSecOps

    Leveraging eBPF for DevSecOps

    eBPF is a revolutionary technology originating from the Linux kernel. It is used to safely and efficiently extend the kernel’s capabilities without requiring changing the kernel source code or loading kernel modules/extensions.

    Today, eBPF is used extensively to:

    • Provide high-performance networking and security in modern data centers and cloud-native environments
    • Extract fine-grained security observability data at low overhead
    • Provide insights for performance troubleshooting
    • Provide preventive application and container runtime security enforcement

    eBPF-Enriched Context

    eBPF is the base data layer that is needed in runtime cloud security. eBPF gives you information at a high level on Layers 3, 4 and 7, allowing you to see all network communication and correlate it with processes and specific applications that are communicating on those ports.

    To fully leverage eBPF, you need to pair this raw data with additional cloud infrastructure context to gain a full understanding of which resources and services are in use.

    For example, by leveraging eBPF, you can determine if you have 10 different applications all communicating on port 443, whereas a traditional network monitoring tool would tell you that you have a certain amount of traffic on this port but wouldn’t be able to segment this traffic and show how much of that traffic is coming from each individual application.

    Knowing which communication is coming from an individual application increases visibility. It allows you to see what your applications are doing in real-time and identify signs of compromise or decreased performance.

    Increasing Observability With eBPF

    eBPF provides users with intrinsic, comprehensive insight into communication flows across the entire system. The applications for enhancing security observability with eBPF are vast, ranging from enabling highly detailed process visibility to facilitating end-to-end observation of processes and flows.

    When dealing with Layer 4 to Layer 7 traffic involving multiple protocols, eBPF emerges as an optimal solution. Through eBPF, TLS traffic traversing pods, nodes and cloud platforms can now be uniformly instrumented across layers.

    In other words, with eBPF, you can observe your application and your node and gain a vast amount of raw data, which, with the right tools, you can correlate to understand your entire network topology.

    Beyond eBPF Raw Data to Context and Correlation: Layer 7 and Beyond

    Specifically for observability, you can use eBPF to apply filters directly in the kernel, which reduces the overhead and allows you to ingest significantly more raw data. eBPF’s architecture allows you to do all of this, observe network traffic and Layers 3, 4 and 7 without compromising your security or performance.

    With eBPF, you are able to understand the execution events, system call events and all of the network ingress and egress. But it’s not always enough.

    You will need to correlate it with Cloud APIs, Kubernetes identities, pods and namespaces to understand the actual identities and relationships between the individual workloads, all the way down to the application layer. By doing so, you are able to understand application-level identity and use this context to understand your application behavior as well as your most critical risks and threats.

    Leveraging eBPF for Threat Detection and Response

    Security and networking observability are inherently interconnected. eBPF helps you to gain insights such as abrupt shifts in behaviors, spikes in activity within specific processes indicating potential exploitation or attacks, service-to-service communication trends and more. These projects enable users to examine network transactions, identifying the pods, processes and system calls involved.

    With eBPF, you can see OS-level runtime threats. eBPF gives you the ability to listen to system calls but also to go one step further and block or terminate any process, PID (process identifier), system call or network packet. This allows for a more granular approach to automated threat detection and response without damaging your system or hurting your infrastructure.

    You can also use eBPF to implement prevention rules, meaning you can instruct eBPF to automatically block any malicious processes or pre-designated system calls or network packets, going beyond traditional uses of eBPF for monitoring and security, to become a more proactive, preventative tool for securing your infrastructure and applications.

    Real-World Benefits of eBPF

    eBPF presents a number of real-world benefits, going beyond traditional monitoring, observability and security to help with overall alert noise reduction and improved forensic analysis.

    How eBPF Contributes to Noise Reduction

    By itself, eBPF does not reduce noise. In fact, it increases because of the influx of raw data from every system call. However, with the right tools, you can leverage this raw data and correlate it with additional context to understand your infrastructure and application behavior. This effectively reduces the alert noise that you would otherwise receive because you can view more detailed insights into resource behavior.

    By filtering and aggregating this data, you receive more insights about applications, overall giving you less noise because you’re focusing on what actually matters.

    By having this eBPF raw data at runtime, you can apply intelligent filters to filter through the most important data and use it accordingly, resulting in better prioritization, more focused efforts and increased operational efficiency.

    Forensic Analysis

    Another important real-world use case for eBPF is conducting forensic analysis. By leveraging eBPF’s raw runtime data and applying intelligent filters, you can quickly pinpoint problem origins and conduct forensic analysis at the application level.

    By leveraging eBPF’s raw data from Layer 7, you gain a granular understanding of applications and the process-level communication within an application, allowing you to foresee the potential blast radius if an application were compromised in terms of who it could interact with and what damage could occur.

    Conclusion

    In summary, eBPF is a powerful technology that can be effectively leveraged for monitoring, observability and security purposes. However, when eBPF’s raw data is combined with context from cloud APIs, it can be leveraged even further with intelligent filters to provide unparalleled cloud infrastructure and application context, allowing you to understand application-layer identity, provide real-time security, pinpoint problem origins and stop threats with surgical, active response.

    To hear more about cloud-native topics, join the Cloud Native Computing Foundation, Techstrong Group and the entire cloud-native community in Paris, France at KubeCon+CloudNativeCon EU 2024 – March 19-22, 2024.

  • Sumo Logic Eliminates Ingest Fees for Observability Log Data

    Sumo Logic Eliminates Ingest Fees for Observability Log Data

    Sumo Logic this week announced it will no longer charge a fee for ingesting log data into its observability platform to encourage DevOps teams to apply analytics more deeply.

    Michael Cucchi, vice president of product marketing for Sumo Logic, said the Sumo Logic Flex Licensing plan eliminates what amounts to a tax for using an observability platform. Flex Licensing is being made available immediately to new customers and will be offered to existing Sumo Logic customers later this year. There are no hidden monthly charges, feature restrictions, performance tradeoffs or user limitations being applied under the plan.

    The licensing plan only applies to log data, but DevOps teams will be able to ingest all structured, semi-structured and unstructured data for a lower cost that Sumo Logic is working to store more efficiently, said Cucchi. The overall goal is to make available a disruptive approach to licensing that lowers the total cost of observability, he added.

    Storing log data is going to be increasingly critical as organizations begin to apply multiple types of artificial intelligence (AI) models to further automate IT operations, he added. Those AI models promise to make observability platforms more accessible because algorithms will be able to surface issues that need to be addressed. Historically, the value of an observability platform has been tied to how proficient a DevOps team became when using its query language.

    IT teams that have adopted observability platforms have been, to varying degrees, limiting the amount of data they collect and retain to keep storage costs from exploding. The challenge is that as DevOps teams deploy more microservices-based applications, the amount of log data generated has exploded. Determining the root cause of any issue involving those applications can be problematic if log data isn’t readily available. Log data is, after all, the atomic unit for observability, noted Cucchi.

    In general, IT environments are becoming too complex for humans to manage without the aid of observability platforms augmented by AI models, noted Cucchi. Each application environment is fairly unique, so it is pivotal for those AI models to be exposed to as much data as possible to ensure optimal results.

    It’s not clear whether observability platforms might one day replace the need for many of the monitoring tools that IT teams rely on today to track a set of pre-defined metrics. However, one thing that is certain is that observability platforms, at the very least, present an opportunity to rationalize some of those tools. The biggest challenge is finding ways to fund the acquisition of an observability platform in the first place, which then creates an opportunity to consolidate those tools later.

    Given the complexity of highly distributed computing platforms, it’s only a matter of time before most IT teams have some type of observability capability in place. Of course, observability has always been a core DevOps tenet. The trouble is that the tools being used to achieve it are legacy monitoring platforms that don’t provide the level of depth required to successfully manage modern application environments.

    Image source: Joshua Sortino via Unsplash.

  • Permiso Makes Open Source Tool to Identify Cloud Threats Available

    Permiso Makes Open Source Tool to Identify Cloud Threats Available

    Permiso today launched an open source tool dubbed CloudGrappler that surfaces indicators of compromise in cloud computing environments.

    Based on a cloudgrep project originally created by Cado Security, CloudGrappler detect patterns that Permiso, a provider of an identity threat detection and response platform, associated with tactics and techniques commonly used by various cybercriminal syndicates.

    The tool enables DevSecOps teams to define the data sources they want to scope in a scan of an Amazon Web Services (AWS) or Microsoft Azure cloud. Via a JSON file, DevSecOps teams can access a list of predefined tactics, techniques and procedures (TTPs) that are commonly used to exploit cloud computing environments. DevSecOps teams can also dynamically add queries or a file with multiple queries to scan a specific data set. After scanning, CloudGrappler delivers a JSON report that provides a breakdown of the scan results.

    Ian Ahl, senior vice president for P0 Labs, the research arm of Permiso, said that capability makes it possible for DevSecOps teams to identify issues in cloud computing environments that warrant further investigation before there is a major incident. The primary goal is to make it simpler for DevSecOps teams to determine if they need additional assistance to thwart a potential threat, he added.

    Cybercriminals are, like almost everyone else, creatures of habit and often tend to leverage the same TTPs multiple times simply because they know they’re effective. However, those TTPs also provide clues to their presence in an IT environment. Once discovered, a DevSecOps team can decide if they can remediate the issue themselves or need to tap external cloud security expertise, noted Ahl.

    In general, cloud computing environments may be more secure than most on-premises IT environments. However, the processes used to provision cloud computing environments are often flawed because developers typically don’t have much cybersecurity expertise. Misconfigurations are often rife across cloud computing environments.

    Cybercriminals are also especially adept at stealing credentials to gain initial access to cloud computing environments, which they leverage to escalate privileges in a way that provides them unfettered access to the entire cloud computing environment.

    Adoption of DevSecOps best practices has improved cloud security, but there are still plenty of instances where, for example, a port inadvertently left open has enabled cybercriminals to infest a cloud computing environment with malware that could lie dormant within cloud applications for months before finally being activated.

    There are simply not enough cybersecurity experts available to secure cloud computing environments, so it’s going to be largely up to DevOps teams to secure them. The issue is that in the race to build and deploy applications as quickly as possible, cloud security best practices are either forgotten or simply ignored. Of course, it’s that very carelessness that cybercriminals are counting on to allow them to wreak havoc in an era where the percentage of mission-critical applications running in cloud computing environments will only continue to increase exponentially.

  • Copado Streamlines Custom Salesforce App Testing

    Copado Streamlines Custom Salesforce App Testing

    Copado today added a testing tool to its platform for managing DevOps workflows in Salesforce application environments that automatically creates scripts based on manual tests being conducted.

    Announced at the TrailblazerDX conference hosted by Salesforce, Copado Explorer enables DevOps teams to integrate those scripts into testing processes that can be repeated as required.

    David Brooks, senior vice president and lead evangelist for Copado, said the goal is to make it simpler for DevOps teams to add those tests to the regression tests that are incorporated into pipelines created using the Copado continuous integration/continuous delivery (CI/CD) platform.

    In addition, markup and annotation tools enhance collaboration by allowing non-technical subject matter experts to provide detailed feedback and insights to quality assurance and development teams as part of an effort to streamline communications between product owners, testers and developers. Testers without a technical background can use that capability to contribute to the creation and deployment of effective automated tests. That capability makes it possible to shift testing further left to incorporate the business teams that better understand how an application should function, said Brooks.

    The challenge is far too many organizations still overlook the importance of testing, noted Brooks. There is a tendency when using low-code tools to assume that not as much testing is required because the tools that generate code create a false sense of instant quality despite flaws that might not manifest themselves until an application is deployed, noted Brooks.

    Rather than constantly updating applications, more testing should improve the quality of the applications being built sooner. That’s critical because most organizations don’t have the time and resources needed to continuously iterate applications until they eventually arrive at what the business requires, said Brooks.

    It’s still early days so far as the level of DevOps maturity that organizations using the Salesforce platform to build and deploy custom applications have thus far attained, noted Brooks. However, just about every organization that licenses the software-as-a-service (SaaS) platform from Salesforce customizes those applications to better align them with their internal business processes, he added.

    Each organization will naturally need to determine to what degree they are prepared to deploy customer applications, but it’s clear it’s becoming simpler to build them. Late last year, Copado, for example, added generative artificial intelligence (AI) tools to make it easier to manage the DevOps workflows used to build these applications, which means the speed at which these apps are built and deployed is about to exponentially increase.

    Salesforce, of course, is only one of many SaaS platforms that organizations rely on to build and deploy custom applications. However, as one of the most dominant platforms, the pace at which custom applications are being built continues to increase, with a larger percentage of these applications being built by professional developers familiar with DevOps best practices.

    Eventually, those DevOps workflows will be extended to every kind of developer, but for now, most organizations building applications are looking for more consistent outcomes regardless of what type of tool is used to build them.