devopsy.kinsta.cloud

Tag: generative AI

How Generative AI Enables Unified Continuous Testing Platforms

It is time for a unified continuous testing platform.

Existing software testing platforms and tools specialize in testing activities for a subset of the different stages of value streams. The landscape of software testing tools includes a wide range of solutions, each with its strengths, focusing on specific aspects of software development, delivery and operations.

For instance:
• Unit Testing Tools focus on the earliest stages of development, allowing developers to test individual units of code for correctness.
• Integration Testing Tools aim to test the interactions between different modules or services within an application.
• System Testing Tools are designed for end-to-end testing of the complete system before it goes live.
• Performance Testing Tools assess the application’s behavior under load and stress conditions.
• Security Testing Tools focus on identifying vulnerabilities within the application.
• User Acceptance Testing (UAT) Tools facilitate the final testing phase, where end-users validate the solution against their requirements.

The challenge with unifying these diverse testing needs into a single continuous testing platform is multifaceted. Such a platform needs to seamlessly integrate with a wide variety of development tools and environments, support different testing methodologies, and be flexible enough to adapt to different organizational processes and quality standards.

While there are Continuous Integration/Continuous Deployment (CI/CD) tools that incorporate multiple testing stages, they often do so by integrating with specialized testing tools rather than offering a unified testing solution themselves. These CI/CD tools are closer to orchestrating the various testing activities rather than unifying them under a single platform’s capabilities.

However, the concept of a fully unified continuous testing platform, covering every stage from requirements through to deployment and in-production testing, represents a significant opportunity for innovation in the software development tools space. Achieving this would likely require leveraging advancements in areas like generative AI, as discussed in my prior blog Applying AI/ML to Continuous Testing to create adaptable, intelligent testing processes that can cover the full spectrum of testing needs in a cohesive manner.

Benefits of a Unified Continuous Testing Platform

Improved Efficiency and Speed: A unified CT platform integrates testing across all stages of the software development lifecycle (SDLC), significantly streamlining the testing process. This integration facilitates automated, continuous feedback loops that promptly identify and address defects, allowing development teams to quickly iterate and improve. As a result, teams can release new features and fixes more rapidly, accelerating the product’s time to market and enhancing responsiveness to customer needs and market changes.

Enhanced Quality and Reliability: By ensuring comprehensive and consistent testing across the entire application, a unified CT platform plays a crucial role in identifying and mitigating issues early in the development cycle. This early detection helps maintain a high level of software quality and reliability, leading to increased user satisfaction and trust in the product. The consistent application of quality standards across all testing phases contributes to a robust and dependable software product.

Cost Savings and Reduced Maintenance: Automating the testing process through a unified CT platform not only optimizes resource utilization but also significantly reduces the costs associated with manual testing and late-stage defect remediation. Early defect detection translates to lower fix costs, and the streamlined process decreases the overall time-to-market. Additionally, high-quality, reliable software requires less maintenance, further reducing long-term costs and freeing up resources for innovation and development efforts.

Improved Collaboration Across Teams: A unified CT platform fosters a culture of collaboration and transparency among development, testing, and operations teams. By providing a common framework and tools for all testing activities, it breaks down silos and enables seamless communication and collaboration across the SDLC. This improved collaboration ensures that teams are aligned with the project goals, can share insights more effectively, and work together to identify and solve problems more efficiently, leading to better outcomes.

Together, these benefits highlight how a unified Continuous Testing platform can transform the software development process, making it more efficient, cost-effective, and collaborative while ensuring the delivery of high-quality and reliable software products.

While many tools address specific parts of the testing lifecycle, the vision of a single platform that seamlessly unifies all stages of testing from requirements to in-production remains an ambitious goal. It’s a gap in the current technology landscape that presents both a challenge and an opportunity for future development.

Challenges for Unified Continuous Testing Platforms

Creating a continuous testing platform that unifies test activities for all stages of the end-to-end value stream is a complex challenge due to several factors:
1. Diversity of Technologies and Tools: Modern software development environments are highly diverse, incorporating various programming languages, frameworks, and technologies. Creating a platform that seamlessly integrates with all these technologies is challenging.
2. Complex Integration Points: Continuous testing needs to integrate with multiple stages of the development pipeline, including development, deployment, and operations. Each of these stages might use different tools and processes, making it hard to create a one-size-fits-all solution.
3. Varying Quality Metrics: Different teams and projects may have different definitions of quality, success criteria, and performance metrics. A unified testing platform needs to be highly customizable to cater to these varying needs.
4. Change Management: Adopting a new platform requires changes in the organization’s processes and workflows. Resistance to change is common in organizations, and transitioning to a new way of testing can be met with skepticism and inertia.
5. Scalability and Performance: Ensuring the platform can scale to handle the testing needs of large organizations with thousands of tests running concurrently is a technical challenge. Performance issues can become a bottleneck, affecting the overall efficiency of the development process.
6. Security and Compliance: Integrating testing across all stages of development also introduces security and compliance challenges. The platform must ensure that sensitive data is protected and that testing practices comply with regulatory requirements.
7. Cost and Resource Constraints: Developing, maintaining, and supporting a unified continuous testing platform requires significant investment. Organizations might be hesitant to commit the necessary resources without clear evidence of return on investment.
8. Evolution of Practices: Software development practices and tools are constantly evolving. Keeping the platform up-to-date with the latest practices and technologies requires ongoing effort and innovation.

Despite these challenges, there is a growing recognition of the value of continuous testing throughout the software development lifecycle. Some companies and open-source communities are making strides in this direction, creating more integrated and flexible testing solutions. However, achieving a fully unified platform that addresses all these challenges is an ongoing effort and represents a significant opportunity for innovation in the software development and testing industry.

Generative AI Can Facilitate a Unified Continuous Testing Platform

Generative AI can play a significant role in overcoming the challenges associated with creating a continuous testing platform that unifies test activities for all stages of the end-to-end value stream. Here’s how generative AI might address each of the challenges:
1. Diversity of Technologies and Tools: Generative AI can be trained on a wide range of programming languages, frameworks, and technologies to understand and generate code or testing scripts. This capability allows it to adapt to different environments and create testing materials that are compatible with various tools and technologies.
2. Complex Integration Points: AI can analyze the workflow of development pipelines and suggest optimal integration points for testing. By learning from different CI/CD (Continuous Integration/Continuous Deployment) configurations, AI can recommend best practices for integrating testing seamlessly into existing workflows.
3. Varying Quality Metrics: Generative AI models can be customized to understand and apply different quality metrics and success criteria based on project-specific requirements. By training on diverse datasets, these models can adapt to various definitions of quality and generate relevant tests or analysis.
4. Change Management: AI can assist in the change management process by simulating the outcomes of adopting new testing platforms, thereby providing evidence-based benefits and mitigating resistance to change. Furthermore, AI-driven analytics can highlight the efficiency gains and quality improvements to support the transition.
5. Scalability and Performance: Generative AI can optimize testing processes by identifying redundancies and suggesting improvements, thus enhancing performance. Additionally, AI can dynamically allocate resources based on testing needs, ensuring scalability without compromising efficiency.
6. Security and Compliance: AI models can be trained to identify and flag potential security and compliance issues in the testing process. By continuously learning from the latest security standards and compliance regulations, AI can help ensure that testing practices meet the necessary requirements.
7. Cost and Resource Constraints: By automating the generation and optimization of test cases, generative AI can significantly reduce the manual effort required, lowering costs and resource demands. AI can also help prioritize testing efforts based on risk assessment, ensuring that resources are focused where they are most needed.
8. Evolution of Practices: Generative AI models are inherently adaptable and can continuously learn from new development practices, tools, and technologies. This ensures that the testing platform remains up-to-date with the latest advancements in software development.

Generative AI has the potential to transform continuous testing by providing adaptive, efficient, and intelligent solutions to the complex challenges of unifying test activities across the end-to-end value stream. However, realizing this potential requires careful design, extensive training of AI models, and ongoing management to ensure that the AI systems remain effective and aligned with evolving testing needs.

Summary: Call to Action

The need for a unified Continuous Testing (CT) Platform has never been more urgent. Current testing platforms, each adept in their niche, cover only fragments of the software development lifecycle (SDLC), leading to a disjointed and inefficient testing process. This fragmentation not only slows down development and deliveries but also compromises the quality and security of the final product. The dream of a single platform that seamlessly integrates all stages of testing, from requirements through to deployment and in-production testing, represents a monumental leap toward efficiency, security, and quality in software development.

The challenges in creating such a platform are manifold, ranging from the diversity of technologies and tools to the evolving nature of software development practices. Each challenge, from integrating diverse technologies and managing change within organizations to ensuring scalability and compliance, adds complexity to the development of a unified CT platform. Yet, the potential benefits of overcoming these hurdles are immense, promising a significant boost in the speed and quality of software delivery. The industry’s recognition of these benefits is growing, evidenced by the efforts of some companies and open-source communities moving towards more integrated and flexible testing solutions.

Generative AI emerges as a beacon of hope in this quest, offering innovative solutions to the multifaceted challenges of unifying test activities. By harnessing the power of generative AI, the industry can address the diversity of tools, integrate complex testing stages, adapt to varying quality metrics, manage organizational change, scale efficiently, ensure security and compliance, and evolve with the practices of software development. The path forward requires a concerted effort, but investments in AI-driven testing innovations can realize the vision of a comprehensive, unified, continuous testing platform. This is not just an opportunity for enhancement but a call to action for the industry to redefine the future of engineering platforms.

March 21, 2024
AISecOps: Expanding DevSecOps to Secure AI and ML

The evolution from traditional software development to integrating artificial intelligence (AI) and machine learning (ML) has been nothing short of revolutionary. As AI adoption continues and AI technologies become essential to businesses and even our daily lives, they are increasingly prime targets for cybersecurity threats.

A particularly alarming trend is the targeting of code and image repositories by cybercriminals aiming to inject malware into the software supply chain. This tactic not only compromises the integrity of the software but also poses a significant risk to end-users and organizations that rely on these applications for critical operations. The threat of data poisoning presents a sinister challenge to the integrity of AI models. By introducing maliciously modified code and data into the training sets, attackers can manipulate the behavior of AI systems, leading to long-term impacts as the poisoned data persists within machine learning models.

This insidious form of attack underscores the importance of vigilance and robust security measures in safeguarding the data that fuels AI and ML innovations. The cybersecurity landscape has been profoundly shaped by our experiences in securing software through DevSecOps practices. These lessons now serve as invaluable blueprints for addressing the familiar challenges posed to AI and ML security, including the need to protect our software supply chain and the integrity of AI models from such insidious attacks.

Over the past five-plus years, DevSecOps has become a staple of how we develop and secure software through collaboration between software and security teams and by embedding improved security practices into every phase of the development process. Is DevSecOps “there yet” as our answer to our software security challenges? No, but it is continuing to improve and evolve as most security practices do. This integrated approach has helped us improve our software products’ security and increased the security visibility between security and software engineers. The principles and successes of DevSecOps can similarly guide the secure development and deployment of AI and ML models.

AI and ML models continuously learn and evolve, making them unique compared to traditional software. AISecOps, the application of DevSecOps principles to AI/ML and generative AI, means integrating security into the life cycle of these models—from design and training to deployment and monitoring. Continuous security practices, such as real-time vulnerability scanning and automated threat detection, protection measures for the data and model repositories, are essential to safeguarding against evolving threats.

One of the core tenets of DevSecOps is fostering a culture of collaboration between development, security and operations teams. This multidisciplinary approach is even more critical in the context of AISecOps, where developers, data scientists, AI researchers and cybersecurity professionals must work together to identify and mitigate risks. Collaboration and open communication channels can accelerate the identification of vulnerabilities and the implementation of fixes.

Data is the lifeblood of AI and ML models. Ensuring the integrity and confidentiality of the data used for training and inference is paramount. Lessons from DevSecOps emphasize the importance of secure data handling practices, such as encryption, access controls and anonymization techniques, to protect sensitive information and prevent data poisoning attacks.

Embedding security considerations from the outset is a principle that translates directly from DevSecOps to AI and ML development. This approach also aligns with the growing emphasis on ethical AI, ensuring that models are not only secure but also fair, transparent and accountable. Incorporating security and ethical guidelines from the design phase helps build trust and resilience in AI systems.

The security challenges presented by AI and ML technologies are complex but not new to us. It’s no silver bullet, but we must utilize the lessons from DevSecOps’ successes and failures. By applying DevSecOps lessons learned to AISecOps, we can approach these challenges with approaches that elevate the visibility of AI and AI data security and emphasize continuous security, collaboration, secure data practices and security by design.

Our future is AI-driven; cybersecurity and AI professionals must come together to fortify the foundations of these transformative technologies. It is critical we unlock the full potential of AI and ML while ensuring the safety, privacy and trust of all stakeholders involved.

March 6, 2024
Google is Right to be Afraid

In the grand scheme of things, at least in relation to the evolution of AI, Google’s recent stumbles come at a rather inopportune time, but the concerns that made them actually address the controversy are worse than most know.

One of the things that generative AI can do really well is answer questions about information available on the internet. While that does not appear to have been the goal–or at least not the only goal–of those who brought us generative AI, it is one of the outcomes. Or, more specifically, replacing search does not seem to have been the goal.

But that is coming fast. While typing a couple of words and getting a list of results is quick and easy–enough so that searching the entire internet has a simple name these days—for many questions, typing a sentence, getting a response, typing some more and getting a refined response is faster. Oh, there is a ton of risk involved in trusting an AI to summarize things for you (as we’ve seen repeatedly, from several providers)—but that risk exists whether it’s AI or not, and is answered by insisting on source links for any fact-based answer. Telling you a story doesn’t need links; telling you about a product absolutely does.

We see that a bit in search engines now, with a short summary at the top that takes from some source the engine decided was authoritative, and the list of links that search engines have always returned below. AI can do this more thoroughly and, when managed responsibly (as opposed to what Google has been doing) can create higher-quality answers than simply quoting the top or “most authoritative” source.

The ability to search will be bolted onto a wide variety of products, meaning that the backend for such search will be what is left to products like Google and Bing. Each misstep by a vendor will drive product creators to want increasing control of their end of the AI, and failure to cater to those needs will drive both the product implementors and customers away from the backend vendors.

My advice? Watch and wait. Try using AI to search; it works really well at returning specific information and, when prompted, links also. This journey is going to be bumpy, and all indications are that the near-term economy won’t tolerate a lot of mistakes. Vendors will come and go, products will be better at one thing and less so at another. And to protect the org, don’t get carried away in the trend to invest in AI for every single product. That is the future, but there are a lot of twists and turns in the trail between here and there.

Note a specific thing that the organization needs help with and with which AI might be able to assist, then go looking for valid solutions. Don’t buy things just because it includes generative AI—have a purpose. And pay attention to what your existing vendors are doing.

Google isn’t even vaguely close to down-and-out; they have more money than most nations. But they are set back a long way, and will have to work to regain trust. For some, there is no path to regain trust, and Google will have to accept that.

And we need to be pragmatic. You work for a business, so make business decisions based on what is best for the business, no matter what others are doing. Keep knocking out astounding apps, and keep smiling—because AI is eventually going to make your job easier.

March 6, 2024
StarCoder2 LLMs Designed Specifically to Generate Code Arrive

ServiceNow, Hugging Face and NVIDIA have once again combined their efforts to advance a generative artificial intelligence (AI) platform that has been specifically trained to generate higher quality code than rival approaches based on general-purpose large language models (LLMs) such as ChatGPT.

StarCoder2 is a family of LLMs created by the BigCode community led by ServiceNow. Three iterations are starting now with a 3 billion‑parameter model trained by ServiceNow; a 7 billion‑parameter model was trained by Hugging Face, while NVIDIA built a 15 billion‑parameter model.

The 3 billion‑parameter model is capable of matching the performance of the original StarCoder 15 billion‑parameter model.

Nicolas Chapados, vice president of research for ServiceNow, said as AI research continues to advance, each successive wave of LLMs becomes more efficient. Each organization, depending on the use case and total cost, can then opt to deploy StarCoder LLMs either on a local machine or in the cloud. The more parameters an LLM has, the more infrastructure resources are needed to run any inference engine based on it.

The StarCoder2 LLMs were trained using 619 programming languages and can be extended to, for example, provide summarizations of code. The foundation of StarCoder2 is a new code dataset called The Stack v2 that is more than seven times larger than version 1 of the dataset. In addition, additional training has enabled the model to understand low‑resource programming languages such as COBOL, mathematics and program source code discussions. Organizations can also fine‑tune models with industry or organization‑specific data using open source tools such as the NVIDIA NeMo platform or Hugging Face transform reinforcement learning (TRL) techniques. StarCoder2 was built using data under license from the digital commons of Software Heritage hosted by Inria.

The overall goal is to provide development teams with LLMs that, in addition to being less likely to hallucinate, also generates code with much fewer vulnerabilities because the code generated is based on exampled vetted by the BigCode community, said Chapados.

It’s not clear right now how much of the code finding its way into production environments is being generated by machines versus humans. However, as LLMs continue to advance, the percentage created by machines will undoubtedly increase. The challenge DevOps teams now face is determining how to apply AI to pipelines that will increase in number as codebases not only proliferate but also become inevitably larger.

In the meantime, DevOps teams would be well-advised to become familiar with what type of LLM is being used to train the generative AI platforms being used to create code. The more general purpose an LLM is, the more likely it is to generate suboptimal code that DevOps teams are only going to need to eventually replace, especially if that code is consuming expensive infrastructure resources inefficiently.

There is, of course, as far as generative AI is concerned, no going back at this point. The issue now is determining how to keep track of the provenance of all the code being generated to determine how best to fix it whenever needed.

March 4, 2024
Logz.io Leverages AI to Identify Anomalies in Real-Time

Logz.io this week revealed it has added a real-time anomaly detection capability to its observability platform that makes it simpler to correlate the impact IT events will have on business processes.

This capability is based on artificial intelligence (AI) models trained using historical data collected via the Logz.io Open 360 platform.

Logz.io CTO Asaf Yigal said this addition to the Open 360 platform enables DevOps teams to employ machine learning algorithms to identify anomalies in mission-critical processes that warrant further investigation. Armed with insights that go beyond simple IT infrastructure consumption metrics, it becomes possible to significantly reduce mean time to resolution, he added.

Logz.io has been embedding AI models within its platform since last year. Still, this latest capability will enable organizations to further bridge a divide between IT and the business that has long plagued many organizations, noted Yigal.

The ultimate goal is to make it simpler for DevOps teams to instantly identify any anomalous application behavior that represents a risk to the business, including cybersecurity events, he added.

The AI capabilities that Logz.io has been steadily adding to its observability platforms are largely transparent to DevOps teams, said Yigal. Instead, Logz.io is embedding multiple types of AI models into its platform that can be invoked in much the same way any other capability is provided, he added. DevOps teams are not necessarily going to need to build their own models to take advantage of AI to observe application behavior at scale, noted Yigal.

The goal is to surface those capabilities consistently at a level of cost organizations can afford, added Yigal.

It’s not likely AI will replace the need for DevOps teams any time soon, said Yigal. After all, there is a reason AI models are referred to as co-pilots versus an automatic pilot, he noted.

It’s not clear to what degree organizations today can correlate the impact an IT event is going to have on the business. But as organizations continue to realize how dependent they are on applications to drive digital processes, the need to bridge the historic divide between IT and the rest of the business has become more urgent. In fact, many of the product and brand champions that organizations now rely on to manage a business unit often have a background in DevOps.

Of course, business professionals that are steeped in IT are still relatively rare but as advances in AI continue to make it simpler to understand how complex application environments function the number of users of an observability platform will naturally expand beyond IT. The challenge is presenting each stakeholder with a view into the IT environment that meets their specific requirements.

It may take a yet before the divide between IT and the rest of the business is permanently bridged but with each passing year it gets narrower. The challenge and the opportunity now is making sure everyone on both sides of that divide understands the art of the possible today that only a few short years ago would have seemed impossible to achieve.

March 1, 2024
404: Role Not Found – Generative AI’s Impact on DevOps Jobs
Many DevOps professionals are wondering if generative AI will be a friend or foe, even as DevOps remains at the forefront when it comes to onboarding new technologies. Our DevOps communities have been enabling and fostering the collaborative cultural shift and evolving roles and responsibilities of software professionals. In this article, we explore the impact of generative AI in key DevOps roles and explain how to prepare ourselves for the shift.

Let us start by highlighting the basic roles and their broad areas of responsibility so we begin with a common vocabulary.

Overview of DevOps Roles

DevOps roles are evolving quickly, but PagerDuty outlined six main DevOps roles for modern organizations.
1. The DevOps Evangelist: Responsible for delivering changes, collaborating with dev and ops teams and working toward optimizing flow, enabling feedback and fostering experimentation.
2. The Code Release Manager: Responsible for creating cadence between delivery and deployment of code. This role often requires deep technical expertise to run and maintain the releases.
3. The Automation Architect: Responsible for designing and implementing automated pipelines and reducing manual toil.
4. The Experience Assurance Expert: Responsible for user experience, enabling feedback loops and ensuring superior experience, assessing feature performance and improving end-product usability.
5. The Software Developer & Tester: Developers and testers are responsible for the design and development of the software product. Primary responsibilities are writing code, testing and fixing bugs.
6. The Security and Compliance Engineer: Responsible for the overall security and compliance of the software product. Collaborating with all the above roles to ensure standards and regulations are implemented.
All roles above are expected to work across the value chain to maximize their impact. Note that there are many other roles, including leadership and executive roles, which we have not covered in this article. Next, let’s assess the impact of generative AI on some of these roles.

Adding Generative AI to Every Role

Before diving into the business value of adding generative AI to job roles, let’s build a common vocabulary, starting with defining generative AI. According to Gartner, generative AI can learn from existing artifacts to generate new, realistic artifacts (at scale) that reflect the characteristics of the training data but don’t repeat it. It can produce novel content such as images, video, music, speech, text, software code and product designs.

Automated code generation: AI-augmented code generation tools serve as companions for developers and testers. As an example, GitHub Copilot, developed by GitHub in collaboration with OpenAI, primarily offers predictive code generation and supports multiple languages. Tabnine offers deep learning capabilities and supports around 20 programming languages.

As these and other tools evolve, software developers’ time may be freed up to focus on higher-level, more creative, and innovative thinking and business value-driven development. Some pundits say these code generation tools may be a stopgap in the race to hire developers. It may also result in generative AI-based moderators, a new job role entirely. These moderators would help with exporting or reusing existing code rather than trying to reinvent it.

AI-based release management: Generative AI can also help with release management; from a simplified approach for generating release notes to boosting the productivity of the feature release to setting a precise release cadence, generative AI-based tools are coming to our rescue. For example, Digital.ai introduced predictive decision-making, threat insights and many more features to orchestrate releases. Another example is K8sGPT, a tool for scanning your Kubernetes clusters, diagnosing and triaging issues in simple English. More integration of generative AI is on the way for DevOps tools. Release managers’ roles are at a pivotal point and will evolve with time and new skills to manage real-time releases, integration with several channels of special edition software, classic upgrades and core software releases. They will need to address different customer segments with managed subscription-based, open source versions and take a human-centric approach to software releases.

Security and compliance engineers: DevSecOps firm Jfrog is integrating generative AI capabilities into its Xray and Artifactory solutions. It will automatically detect security flaws, license violations and more early and often. Snyk surveyed more than 400 IT professionals and found the use of artificial intelligence (AI) to write code is creating a security paradox. Generative AI also can be used to simulate malware attacks and identify vulnerabilities. However, this is a double-edged sword as it also can be used to enhance the portfolio of bad actors. That means that security and compliance engineers must increase their focus on regulatory, assessment, audits and validation of generative AI outcomes. Specialist knowledge, decision-making, and ethical evaluation are needed for a strategic approach toward next-generation cybersecurity capabilities.

The automation architect: An automation architect is responsible for decomposing complex tasks and automating them through reusable code. These tasks become more sustainable, with less rework and delays. As an automation architect, generative AI is another tool in the toolset that helps improve efficiency and productivity. Generative AI can help automation architects to scale and allow self-service options for DevOps teams and automate redundant tasks.

Generative AI’s Impact on Key Job Roles

As discussed in the previous section, there are many jobs that will be transformed by generative AI’s influence. Many companies will view generative AI-enhanced tools as a pot of gold where most of the work that requires building templates, infrastructure provisioning, log analysis, reuse or refactoring code can be done with the help of generative AI-enabled tools.
- Short-term skills gap: Software-related skills gaps are everywhere, and organizations need to go beyond reskilling to solve the problem. The role of leaders is key for successfully transitioning into the new era of workforce modeling. DevOps professionals should also look at strategies to improve our productivity by investing in capabilities and tools to fill the skills gap.
- Mid-term talent augmentation: Role disruption is most likely to happen due to generative AI; research and a more structured approach are needed to address the upcoming change. DevOps teams must focus on communication patterns, onboarding of human-machine collaborative models and ensuring performance of the new work models. Copyright, intellectual property (IP), licensing and the use of open source technology will be essential ingredients of mid-term talent augmentation with generative AI.
- Long-term, flexible workforce: Widespread and massive onboarding of generative AI in the long term will require a pragmatic approach to flexible workforce modeling. Task-based frameworks will emerge to guide the flexible orchestration of workforce, roles or even jobs or tasks. Task-based frameworks may mature into capability-driven roles that evolve into a flexible workforce where humans and machines together will deliver the desired business outcomes.
With rapid advancement in generative AI technology, companies will be forced to design new ways of working to keep pace with these changing dynamics. DevOps professionals should see this as an opportunity to get ahead of the curve, assess how generative AI technologies can boost productivity and enhance it with new tools and capabilities.
February 26, 2024
Tabnine Extends Generative AI Testing Platform by Embracing RAG

Tabnine today revealed that its namesake generative artificial intelligence (AI) platform for creating test code can now surface more accurate and personalized recommendations based on specific code and engineering patterns.

At the same time, Tabnine also announced that Tabnine Chat, a tool for interacting with the large language models (LLMs) at the core of the Tabine platform using a natural language interface, is now generally available.

Data that resides in a codebase or an integrated developer environment (IDE) can now be used to extend the large language models (LLMs) on which the Tabnine platform is based using retrieval augmented generation (RAG) techniques. Previously, the Tabnine platform would only generate test code based on the data that Tabnine had exposed to its LLMs.

Tabnine president Peter Guagenti said RAG makes it possible to provide additional contextual awareness for the test code generated along with documentation and explanations of the test code created. The overall goal is to leverage generative AI to make it easier to shift more responsibility for application testing further left toward developers, he noted.

That approach enables developers to create and run more routine tests earlier in the software development life cycle, which should provide application testing teams with more time to run more complex tests before applications are deployed, added Guagenti. Rather than being a job killer, generative AI will eliminate rote tasks that most members of the DevOps team don’t really enjoy doing, he noted.

Tabnine uses a mix of LLMs it developed with Google to apply generative AI to application testing. Unlike general-purpose LLMs, the ones used by Tabnine were trained using a narrower base of testing code that the company curated using tests made freely available on the internet.

It’s now only a matter of time before generative AI is applied across the entire software development life cycle (SDLC). It’s not clear how automated testing will become in the age of AI, but the overall quality of applications should steadily improve as more tests are run. The goal is to make it possible to conduct more tests without slowing down the overall pace of application development.

There continues to be as much irrational exuberance about AI as fear and loathing. Still, DevOps teams that committed early on to ruthless automation will naturally be at the forefront of adoption. The immediate challenge is assessing what functions will be automated and the impact those changes will have on the way DevOps teams are currently structured. Ultimately, the goal is to eliminate as many of the bottlenecks that conspire to slow application development and deployment while simultaneously improving the quality of the application experience.

It may be a while before generative AI is pervasively applied across DevOps workflows, but one of the first places it will undoubtedly manifest is in test automation. The challenge and the opportunity now is determining how best to apply it in a way that augments DevOps teams that arguably have never had enough time to properly test code in the first place.

devopsy.kinsta.cloud · No No-Shift, DevOps Jobs, AI Tests Your Code, DevOps End Goal

February 22, 2024
Cycode Brings Generative AI to App Security Posture Management

Cycode today added generative artificial intelligence (AI) capabilities to its application security posture management (ASPM) platform to make it simpler for DevSecOps teams to identify the root cause of vulnerabilities in complex distributed computing environments.

At the same time, Cycode is adding plugins for VS Code and JetBrains integrated development environments, a command line interface and pull request scanners to make it easier to identify security issues as early as possible during the software development life cycle (SDLC). Cycode also revealed today it now has more than 75 connectors to various security tools commonly used within DevSecOps workflows.

Amir Kazemi, director of product marketing for Cycode, said the company is using OpenAI’s generative AI platform to provide a natural language interface for invoking the Risk Intelligence Graph that is at the core of the Cycode ASPM platform. As part of that effort, Cycode is now also making available an Executive Dashboard that, based on all the dependencies uncovered, provides summaries of risks that make it easier to discern what is occurring across an application environment.

The platform, as a result, can now provide automated risk scoring for vulnerabilities based on actual severity and potential impact to the business, said Kazemi. The overall goal is to make it simpler to secure software supply chains by facilitating the proactive collaboration between application developers, DevOps engineers and cybersecurity professionals, he added.

It’s not clear what impact generative AI is going to have on how DevSecOps workflows are constructed, but making it easier to identify the root cause of security issues in application environments made up of monolithic applications, microservices and event-driven frameworks is now essential. Application security has simply become too chaotic to effectively manage without help from AI, noted Kazemi. In its current state, application security is simply unmanageable, he added.

The challenge is that just because a zero-day vulnerability has been discovered, it doesn’t always follow that application development teams know where every instance of it exists within their application environment. It’s not uncommon for DevSecOps teams to discover outdated versions of components that have known vulnerabilities to be running months after a patch in an updated version of that component has been made available.

DevSecOps teams are, of course, always trying to strike a balance between the level of threat a vulnerability represents and the possibility that the patch that should be applied might break an application. However, given the risk to application environments, it’s becoming less risky to patch an application than it is to hope that cybercriminals don’t exploit a known vulnerability.

While much DevSecOps progress has been made in recent years, it’s clear there is still much work to be done. Hopefully, generative AI will make it simpler for application development teams to address issues more rapidly than ever. In the meantime, however, DevSecOps teams should assume cybercriminals are looking to exploit the same AI capabilities to discover vulnerabilities they can exploit faster than ever.

February 21, 2024
Skillsoft Taps ChatGPT to Improve Developer Training

Skillsoft today revealed it plans to add generative artificial intelligence (AI) capabilities to its Codecademy online training platform to provide developers with hands-on experience using ChatGPT to write code.

Kunal Ahuja, general manager for Codecademy consumer business at Skillsoft, said the goal is to give developers a more curated path toward teaching them how to define prompts that will result in better code being generated. These enhancements will be made available to Codecademy Plus and Pro users as well as enterprise customers in the coming months, he noted.

There’s no doubt that developers are relying more on generative AI to write code, but not all the output generated by platforms that are based on a general-purpose large language model (LLM) is going to be consistent. Codecademy provides a more immersive experience that enables developers to refine their prompt engineering expertise, said Ahuja.

AI-enabled capabilities include code explanations and project hints to improve debugging skills, along with other real-time feedback surfaced via a virtual coding assistant. There are also courses such as “Prompt Engineering for Software Engineers” available.

Finally, Skillsoft is also providing access to a mock interview simulator that developers can access to help determine if they are ready for a job interview.

As it becomes more apparent that prompt engineering is a skill application development teams will need to master to build and deploy software, there’s a clear need to teach nearly everyone in IT how to craft prompts efficiently. Ultimately, the number of prompts used to generate code will have an impact on the total cost of building software.

In the longer term, Skillsoft plans to build its own LLM using data it collects via its platform to provide training using an LLM that has been trained using code that has been curated, said Ahuja. One of the issues with a general-purpose LLM is that it is trained using examples of code pulled from across the web that varies widely in quality. That can result in outputs of code that have known vulnerabilities or that simply run inefficiently. Of course, it may be just as likely that a developer augmented by generative AI is, depending on their level of expertise, actually delivering higher-quality code.

Despite those concerns, however, it’s still important to provide developers with the training they need today to remain competitive, said Ahuja.

It’s not clear how much code these days is being generated by machines versus humans, but as the volume increases, the number and size of the software builds that need to be managed is only going to increase. Hopefully, as much AI will be applied to the DevOps workflows used to manage those software builds as there is being applied to writing code, but in the short term, at least, DevOps teams might soon find themselves overwhelmed.

The one thing that is certain is DevOps teams have a vested interest in making sure the code being written by developers is of the highest quality possible. After all, whenever there is an issue with that code, it usually falls to the DevOps team to fix it.

Image source: https://vecteezy.com/programming-background-with-person-working-with-codes-computer-e1679047506780.jpg

February 20, 2024
Harnessing Generative AI for Feature Management Testing
When it comes to DevOps, the emerging integration of generative AI into feature management testing marks a significant evolution. We’re going to take a no-nonsense look at how this technology is revolutionizing the way we create testing environments.

Using artificial intelligence to generate tests allows us to mimic a vast array of user scenarios and environments, which means we can develop and deploy features that aren’t just good but great—robust, efficient and ready for whatever the real world throws at them.

Embracing Generative AI in DevOps Testing

Artificial intelligence is dramatically transforming the world around us, impacting a wide variety of industries at breakneck speeds. A stunning 83% of companies assert that AI is a critical component of their business strategies, particularly when it comes to software development.

Generative AI has quickly become a transformative tool when it comes to DevOps, particularly when it comes to feature management testing, by leveraging machine learning algorithms to generate data and test scenarios that closely mimic real-world conditions.

Traditionally, DevOps teams have relied mostly on manual and a couple of automated testing methods to validate their features. However, these methods often fail to accurately simulate the wide variety of complex and unpredictable user interactions and environmental variables.

By creating nuanced and diverse test cases, generative AI can fill this gap, enabling a more thorough evaluation of features under various conditions. Even now, on a smaller scale, WordPress site owners are using predictive analysis and even simple comparisons to properly orchestrate the rollout of new features–with over 60,000 plugins to choose from, generative AI has the potential to prevent lag, which can often be the result of a single shoddy plugin.

Simply put, AI is actively improving how features are implemented and tested across various platforms by intelligently crafting diverse test scenarios.

Enhancing Test Coverage and Efficiency

The significant expansion in test coverage is one of the key benefits of integrating generative AI into DevOps testing. Traditional testing methods are typically limited by the scenarios that testers can foresee and manually script.

In contrast, AI-driven testing tools can generate a multitude of unique, unexpected test scenarios, capturing edge cases that might otherwise go unnoticed, which is crucial in identifying potential bugs and vulnerabilities early in the development cycle, saving time and resources in the long run.

Additionally, AI can automate both the creation and implementation of these tests. This simplifies the testing workflow, allowing human testers to dedicate their full attention to more intricate and challenging tasks.

Simulating Real-World Scenarios

Another key advantage of generative AI in DevOps is its ability to simulate real-world user behavior and environments.

AI models can create realistic and diverse user scenarios by analyzing historical data, user patterns, and environmental variables. This type of functionality is a huge asset in feature management, where understanding how new features will perform in the real world is paramount.

To use a couple of examples, an AI model can generate test cases that mimic peak usage times, varying network conditions, or different user demographics. Furthermore, companies like Netflix, Google and IBM are already actively using generative AI to simulate real-world scenarios for various applications, such as content delivery optimization, predictive maintenance, and software testing.

Accelerating Feature Development in DevOps

The integration of generative AI into feature testing also accelerates the feature development and deployment process.

A well-constructed prompt chain can set up an environment where DevOps teams can effectively compare different products/services, especially when it comes to more sensitive software, such as cloud management platforms and even find the best ways to escape vendor lock-in without actually wasting time manually comparing and testing everything.

Applying generative AI can also drastically cut down on the time it takes to validate a new feature by generating and running through thousands of test scenarios in quick succession. In DevOps environments, where rapid iteration and continuous deployment are key, this is essential.

AI-driven testing can also provide developers with immediate feedback, allowing for quick adjustments and improvements. This ensures that features are developed faster while meeting the highest quality standards before proper deployment.

Potential Challenges and Considerations

While the potential of generative AI in DevOps testing is immense, there are challenges and considerations that teams must navigate and work to overcome to successfully apply this powerful new tool.

One key issue is the quality and relevance of the data used to train AI models; since inaccurate or biased training data can lead to suboptimal or even harmful outputs, carefully selecting and curating data is a crucial step.

Incorporating generative AI into established DevOps processes can also present challenges in terms of complexity and resource demands. Organizations may face significant challenges in modifying their existing pipelines and processes to accommodate AI technologies, which often require substantial investments in hardware, software and skilled personnel.

Lastly, the need to continuously improve and update AI models cannot be overstated. As new technologies and security threats emerge, these models must be regularly refined and adapted to remain effective and relevant, which will require ongoing investment in AI systems and a commitment to staying abreast of evolving technologies and methodologies in what is a rapidly evolving and advancing field.

Future Directions of Generative AI

Looking forward, the role of generative AI in DevOps is poised to grow even more prominent, and as AI models become more sophisticated and data-rich, their ability to generate realistic and complex test scenarios will only improve.

But let’s look beyond just creating testing environments and isolated testing environments: A well-executed, AI-aided process can reduce costs during SAP staff augmentation and can also streamline processes in other high-level enterprise systems such as Oracle ERP and Salesforce CRM. Overall, ERP software is a perfect candidate for DevOps feature management, as it allows devs to use generative AI for everything from writing code to custom tests.

We can also expect to see further integration of AI in other aspects of DevOps, such as monitoring and performance optimization. The ultimate goal is to create a seamless, AI-enhanced pipeline that tests and deploys features efficiently and continuously learns and improves from real-world feedback.

To better illustrate this transformative impact, some key examples of how generative AI will further impact DevOps in the future include the following:
- AI-Augmented Development: Generative will assist developers with code suggestions, bug fixes, and even making architectural decisions, thereby streamlining the development workflow.
- Improved Infrastructure Management and Automation: The use of predictive analytics, driven by generative AI, will play a crucial role in anticipating infrastructure requirements, facilitating preemptive scaling, and efficient distribution of resources.
- Adaptive Problem Resolution and Incident Handling: Generative AI is set to revolutionize the way operational problems are identified, diagnosed, and resolved swiftly, greatly boosting the dependability and steadiness of DevOps practices. This is evident in platforms such as Google Cloud’s operations suite, where such AI capabilities are already making a significant impact.
- Generative AI in Proactive DevOps: This emerging trend focuses on leveraging generative AI to anticipate potential issues before they manifest, which enables preemptive maintenance strategies and is aimed at reducing system downtime.
- AI-Driven Testing and QA: AI-powered testing will become more sophisticated, with capabilities such as automated test case generation, test data creation, and advanced anomaly detection.
Harnessing Generative AI in DevOps

Integrating generative AI into DevOps is a groundbreaking development poised to revolutionize how we approach software development, testing, and infrastructure management.

This innovative technology offers immense potential to automate complex tasks, enhance efficiency and improve the quality and reliability of software products. However, as with any emerging technology, its adoption comes with its own set of challenges and considerations.

Enterprises aiming to leverage generative AI within their DevOps practices must address challenges associated with data integrity, intricate integration processes, adherence to privacy laws and the necessity for ongoing enhancement of their AI models.

Despite these challenges, the benefits of adopting generative AI in DevOps are quite substantial, ranging from increased efficiency and cost savings to enhanced scalability and improved overall software quality.
February 20, 2024