devopsy.kinsta.cloud

Tag: generative AI

Embracing the Future: Navigating the Waves of AI in DevOps

In the rapidly evolving digital landscape, the advent of generative AI and large language models (LLMs) has ushered in a new era of innovation and transformation. As IT organizations pivot toward DevOps, DevSecOps and SRE mastery, leaders find themselves at the crossroads of excitement and anxiety. This dual sentiment stems from the vast potential of AI to revolutionize operations and the inherent challenges that accompany generative AI’s integration. In this article, we’ll explore the reasons behind this excitement and anxiety, outline the top use cases for generative AI and provide strategic guidance to ensure a smooth DevOps and digital transformation journey.

The Catalysts of Excitement

1. Enhanced Efficiency and Innovation: Generative AI, with its ability to automate complex processes and generate new ideas, promises unprecedented efficiency. It enables IT organizations to streamline development pipelines, reduce manual errors and foster innovation, making the leap toward DevOps and DevSecOps not just a goal but a tangible reality.
2. Improved Decision-Making: Large language models offer sophisticated data analysis capabilities, turning vast datasets into actionable insights. This empowers leaders to make informed decisions rapidly, optimizing operations and enhancing service reliability in line with SRE principles.
3. Competitive Advantage: In the digital age, staying ahead means adopting the latest technologies. Generative AI provides a unique competitive edge, allowing organizations to deliver superior digital experiences, innovate products and services and respond to market changes proactively.

Top Three Generative AI Use Cases in DevOps

1. Automated Code and Test Generation and Review: AI-driven tools can generate code and test snippets, conduct code reviews and testing and suggest optimizations, accelerating development cycles and improving code quality.
2. Security Enhancement: By integrating AI into security protocols, organizations can predict and mitigate potential threats more effectively, enhancing their DevSecOps initiatives with proactive rather than reactive measures.
3. Incident Management and Resolution: AI can automate incident response workflows, predict outages before they occur and suggest remediations, aligning with SRE goals of maintaining high availability and reliability.

Sources of Anxiety

1. Ethical and Privacy Concerns: The potential for misuse of AI technologies, including privacy violations and biased decision-making, raises significant ethical questions, creating apprehension among leaders.
2. Skill Gaps and Workforce Impact: The shift toward AI-centric operations necessitates new skills and roles. Leaders worry about the existing workforce’s ability to adapt and the challenge of filling emerging skill gaps.
3. Integration and Compatibility Issues: Integrating AI into legacy systems and ensuring compatibility across the digital infrastructure poses technical challenges, complicating the transition to advanced DevOps and SRE practices.

Navigating the Digital Transformation Journey

To harness the benefits of AI while mitigating its risks, leaders should consider the following strategic actions:

1. Foster an AI-Ready Culture: Cultivate a culture of continuous learning and adaptability. Encourage your team to embrace AI tools and methodologies, emphasizing the importance of upskilling and reskilling.
2. Implement Ethical AI Guidelines: Develop and adhere to ethical guidelines for AI use, focusing on transparency, accountability and fairness. Ensure privacy and security measures are integral to your AI initiatives.
3. Invest in Talent and Training: Address skill gaps by investing in training programs and partnerships with educational institutions. Consider hiring AI specialists to bridge the gap between traditional IT roles and AI requirements.
4. Prioritize Seamless Integration: Adopt a phased approach to AI integration, ensuring compatibility with existing systems. Leverage APIs and microservices architectures to facilitate smoother transitions.
5. Establish Governance and Oversight: Create a governance framework to oversee AI initiatives, ensuring alignment with organizational goals and compliance with regulatory requirements. This should include monitoring AI performance and its impact on operations and workforce dynamics.

Summary: Seize the AI Opportunity

As we stand on the brink of a transformative era, the fusion of AI with DevOps, DevSecOps and SRE practices represents a significant leap forward. While the journey is fraught with challenges, the potential rewards are immense. By embracing AI with strategic foresight, ethical consideration and a commitment to continuous improvement, IT leaders can propel their organizations toward unparalleled efficiency, innovation and competitive advantage.

In conclusion, the integration of AI into DevOps to further accelerate digital transformation is not just an option but a necessity for staying relevant in the digital age. The journey requires careful navigation, but with the right approach, IT organizations can emerge stronger, more agile and better equipped to face the future. Let’s embrace this opportunity to redefine the digital landscape, ensuring our transformations are not just successful but sustainable and responsible.

Embrace the future; embrace AI.

February 20, 2024
Will Generative AI Succeed Where AIOps Failed?

For decades, the tech industry has been enamored with artificial intelligence, the idea that machines can think like humans. Because these machines never get tired — and compute power increases 10x every five years — we expected them to quickly figure out the meaning of life, the universe and everything else. Around 10 years ago, companies such as Moogsoft emerged, claiming that they could make the lives of IT operations teams a whole lot easier by applying AI to the digital exhaust fumes emitted by infrastructure and applications. Blessed by leading analysts, the segment quickly became known as “AIOps.”

The prevailing thought behind AIOps was that patterns in how applications and infrastructure operate could, over time, be determined — and then used to predict when a problem would occur. Even better, maybe problems could even be fixed before anyone noticed. Magic.

Like most magic, though, not everything is as it appears.

Machine learning is like having 1,000 first graders at your disposal. They’re smart enough to be taught to identify a spike in a chart—or a kitten—and alert you when they see one. However, enterprise environments are inherently noisy. If you ask to be notified every time your first graders see a kitten (spike), get ready to receive a lot of notifications. This simple example highlights why ML is great for identifying kittens in consumer search but not so great for identifying problems with enterprise applications.

What makes matters worse is that modern applications change every day, which means that there will be no predictable pattern in their operation. Put another way, the digital exhaust the application emits cannot be used to create a useful model to detect anomalies.

Most AIOps projects became consulting exercises in writing complex alerting rules. Often, they were great at detecting and predicting things you already knew about but singularly useless in helping with “unknown” problems—the problems you’ve never seen before—and the things SREs troubleshoot in distributed applications and struggle with daily.

The Dawn of Generative AI

Just as the sun appeared to be setting on AIOps, we witnessed the dawn of generative AI. AIOps and generative AI both have the acronym ‘AI’ in them, but that’s where the similarity ends. Generative AI uses large language models (LLMs) to better interpret—and respond to—inquiries. The humble chatbot, aka ChatGPT, quickly became the killer app. Powered by LLMs, ChatGPT talked sense most of the time on almost any topic.

Of course, just like with AIOps, it’s easy to jump to magical conclusions regarding the capabilities of this new technology. ‘Why can’t I just train an LLM on my telemetry data and then ask ChatGPT what the problem is?’ That would imply that telemetry not only had the necessary structure—which it doesn’t—but also that LLMs can reason. Which they can’t.

Let’s dial back the expectations from magical to practical.

LLMs are amazing at interpreting natural language requests and returning coherent results. Every product should, by default, use a GPT-based interface to enable users to access help. In addition, most observability tools use a proprietary scripting language. Every product should also provide a “copilot” so users can generate scripts from natural language inputs. The combination of GPT Help and copilots can provide a 50% boost in productivity to new users.

But what about helping users with the troubleshooting process? LLMs are statistical engines. The next word in a sentence is calculated based on prior words generated and what the most likely next word should be. Imagine applying that to incidents instead of sentences. Instead of generating the next word in a sentence, perhaps you could generate the next step in the troubleshooting flow. That would mean SRE teams could benefit from dynamically created runbooks—always up-to-date and based on the latest and greatest insights.

And we could take it one step further. Instead of simply creating a runbook, what about executing the steps in the runbook before the SRE engages in the incident? Imagine coming to an incident and being presented with a custom dashboard for the specific incident that contains all the data necessary to conduct the investigation. That alone saves an hour in most enterprise troubleshooting flows without promising any “magic.”

AIOps promised a deterministic answer to a non-deterministic problem. It just wasn’t possible to analyze terabytes of machine-generated data and accurately predict a problem with an application, let alone determine the root cause. And generative AI won’t do that, either.

But generative AI can go to so many places that AIOps could never. It provides a general-purpose approach that can be applied in many different ways. Generative AI will lower the barrier to using practically any new tooling, making it easier for everyone to perform common tasks and get up to speed on new scripting languages. Heck, you might never have to write a regex or look up an error message ever again! In addition, LLMs can statistically determine the best next step and, therefore, should be able to make troubleshooting flows more repeatable while benefiting from the most current incident knowledge.

Yes, the SRE still plays a huge role. Of course, LLMs can’t answer questions the underlying tooling is incapable of answering. But I’ll still make the bold statement that five years from now, the average MTTR will be half of what it is today.

February 15, 2024
Pilot or Telephone Operator? Forecasts of Software Developer Demise Are Absurd

Stability AI CEO Emad Mostaque recently grabbed headlines with his audacious prediction that “there [will be] no programmers in five years.” While such statements make great social media clickbait, they don’t accurately reflect the reality of creativity’s role in complex software development. Yes, AI will profoundly change the software engineering profession, but those changes will mostly enable inventive and capable developers to reach new heights of innovation. Software development is already being transformed by generative AI. Microsoft’s GitHub Copilot has more than one million paying customers. Some researchers forecast that between 50% and 80% of software engineering job descriptions will involve generative AI within a few years.

For the foreseeable future, generative AI’s principle value will be automating the mundane tasks that take up so much developer time. Copilots will likely replace many frameworks and code exchanges because they’re faster. They’ll dramatically improve auto-completion and spell check. That’s the grunt work of software development that most programmers will be happy to shed.

Generative AI is expected to be applied most often in testing, vulnerability management and writing code, according to an IDC survey of U.S. developers. However, the same survey found that fewer than 10% of developers expect the technology to have value in such creative disciplines as interface design, prototyping or translating user requirements into applications.

Where LLMs Fail

Building enterprise-grade software requires a keen understanding of business concepts and the ability to design novel solutions. That’s where large language models stumble. Trained on a large corpus of existing data, they excel at quickly spitting out what is already known. That can be of great value, but rewording and repetition aren’t the same as creativity.

Generative AI’s main impact on software development will be reducing mundane tasks and giving developers more time to innovate. Automating manual coding may help dent the chronic shortage of software engineers that shows no sign of easing. However, I have not seen any evidence that technology can think outside the box.

Doom and gloom predictions about automation’s impact on the workforce are nothing new. Occasionally they come true. Thanks to automated switching and direct dialing, telephone operator jobs in the U.S. have fallen by more than 95% since the 1950s. Robots continue to reduce the need for humans to do work no one wants to do.

Up the Stack

The more common outcome is for automation to move jobs higher up the value stack.

● E-commerce and customer self-service were expected to doom the retail industry, but total retail employment is expected to grow nearly 8% through 2030, led by services such as home delivery and specialty merchandise.
● About 90% of a commercial airline flight is now fully automated, yet the shortage of pilots is at an all-time high. That job has evolved from flying the plane to overseeing complex systems and being available for rare but critical emergencies.
● Online banking was supposed to kill bank branches, but there are more today than in 2000. Their role has shifted from processing transactions to delivering complex and customized services to the benefit of tellers and customers alike.
● While travel agent employment dropped substantially with the advent of online booking and the end of airline commissions, the industry is growing again, according to the BLS. Many travel agents have thrived by reinventing themselves as experienced consultants and simplifying the complexities of navigating multiple booking sites.
● Healthcare technology has made huge leaps in diagnostics and patient record management, but the need for human judgment, empathy and complex decision-making ensures that the demand for clinicians will remain strong for a long time.

Is the job of a software developer more like a telephone operator or a commercial airline pilot? Any technology executive will tell you the question is absurd. As a founder and CEO, I believe a top-notch full-stack engineer is more valuable than millions of dollars in venture capital. We’ve only scratched the surface of how software can enrich our lives. Automation will free up developers to spend more time on the tasks that make a difference.

February 15, 2024
Legit Security Adds Sensors to Detect Usage of Gen AI Tools to Write Code

Legit Security today announced it has added the ability to detect when developers use generative artificial intelligence (AI) tools to write code to its application security posture management (ASPM) platform.

The ASPM platform developed by Legit Security already has discovery capabilities for application secrets, for example, enabled by sensors that are now being extended to include generative AI tools.

Legit Security CTO Liav Caspi said the overall goal is to allow DevSecOps teams to identify who uses these tools to write which types of code. Each DevSecOps team can then decide how stringently to enforce policies to either ban or limit usage of those tools, he added.

There is no doubt that generative AI tools are increasing developer productivity, but the large language models (LLMs) that provide the ability to generate code were trained using examples of code of varying quality collected from across the web. Much of that code either contains known vulnerabilities or is simply inefficient, which can manifest itself in the output surfaced by a generative AI platform.

The Legit Security ASPM platform, in addition to identifying generative AI tools being used in real-time, can also be employed to scan the LLM itself to identify security risks such as prompt injection and insecure output handling. Unfortunately, most cybersecurity teams are unaware of the extent to which application code generated by a machine might be insecure, so it currently falls to leaders of DevSecOps teams to determine what is an acceptable use of these tools. It may be perfectly fine to rely on a generative AI tool to create a script that will be used for an internal-facing application. However, using the same tool to write business logic that will be exposed to the internet may not be especially prudent.

It’s not clear how much of the code finding its way into software builds is being generated by a machine versus a human. In the longer term, the goal is to be able to identify that code by, for example, detecting substantial changes in the way code is normally written by a human developer, said Caspi.

Of course, while an ASPM may detect usage of generative AI tools that are connected to a corporate environment, there are still plenty of opportunities for developers to use these tools offline to create code that can later be added to a build using a code check-in tool. After all, many developers have used various shadow IT tools to write code for decades. Regardless of who or what wrote that code, however, it should still be reviewed.

In the meantime, the overall size of the codebase that DevSecOps teams will need to manage and secure will only exponentially increase as developers become more productive. Hopefully, more AI capabilities will soon be embedded into DevOps platforms to further automate the management of software builds at a time when many DevOps teams were already finding themselves overwhelmed even prior to the rise of generative AI.

February 13, 2024
Cisco Extends Scope and Reach of Observability Platform

Cisco this week expanded the scope of its observability platform by adding predictive and generative artificial intelligence (AI) capabilities along with log analytics and a Core Web Vitals tool to make it simple to ensure web pages don’t lose their search engine ranking because of poor user experience.

In addition, Cisco is now leveraging extended Berkeley Packet Filters (eBPF) in the kernel of Linux operating systems to provide visibility into Kubernetes workloads and a session replay capability for cloud-native applications. In addition, Cisco has allied with Komodor, a provider of a Kubernetes management platform, to gain additional visibility into these environments.

Cisco also added digital experience monitoring (DEM) capabilities based on real user monitoring (RUM) and the full-stack observability it provides following its recent acquisition of Accedian, a provider of an analytics platform for monitoring end-user experiences.

Finally, Cisco added a Data Security Posture Management (DSPM) Observability module to the company’s Business Risk Observability platform based on the same core platform.

Announced at a Cisco Live EMEA event, these latest updates are part of an ongoing Cisco effort to extend the reach of its observability platform well beyond DevOps workflows.

Gregg Ostrowski, executive CTO for Cisco AppDynamics, said as part of that effort, Cisco is making it simpler to aggregate telemetry data for the AI for IT operations (AIOps) capabilities it provides by pulling directly from platforms such as Cisco ThousandEyes and Cisco DNA Center as well as from third-party partners such as VMware, Zabbix and ServiceNow. Similarly, Cisco is now leveraging eBPF to collect data from Kubernetes environments.

Ostrowski added that the overall goal is to reduce the amount of instrumentation effort that IT teams would have to undertake to instrument highly distributed IT environments.

There will still be a need to instrument applications, but the collection of telemetry data from the underlying IT infrastructure environment is going to be increasingly automated, he noted. In the meantime, it’s become less expensive to instrument applications using open source OpenTelemetry agent software, added Ostrowski.

In general, Cisco, in addition to enabling organizations to rationalize existing tools, is also making a case for improving collaboration by, for example, making use of natural language capabilities enabled by generative AI to launch queries and provide summarizations.

At the same time, Cisco is partnering to add additional capabilities, such as the ability to monitor machine learning algorithms via an alliance with Aporia and AS/400 systems via an alliance with Perform IT.

Ultimately, Cisco is betting that observability across the entire enterprise IT environment is going to become more unified as organizations increasingly look to correlate business processes with specific IT events, including indications of cybersecurity breaches. It’s not clear how far down that path enterprise IT organizations are just yet, but given the total cost of observing and monitoring IT environments today, there is more pressure than ever to find more cost-effective approaches to holistically managing IT. The challenge, as always, is unifying all the existing fiefdoms in IT organizations that have already embraced a wide variety of tools and platforms they are hesitant to give up.

February 9, 2024
10 Key Features of AI Code Assistants

One of the most potent tools that have emerged to streamline the coding process is code assistants. Code assistants are software tools designed to assist programmers by providing suggestions and automated features that help in writing code more efficiently and accurately. They act as virtual aides, making the coding process faster, more intuitive and less prone to errors.

Code assistants can be categorized into two types: Rule-based and AI-based. Rule-based assistants follow predefined rules and perform according to set guidelines. They are mainly used for syntax highlighting, code formatting and basic error detection. AI-based code assistants can learn from past coding patterns, predict what a programmer is trying to achieve and suggest appropriate code snippets. They are capable of understanding the context of the code and providing accurate suggestions, as well as generating entire blocks of code based on natural language instructions. Read this blog post for examples of popular AI coding assistants.

Whether you are an experienced software developer handling complex projects or a beginner, coding assistants can automate the tedious aspects of coding, allowing you to focus more on problem-solving and innovation.

The Growing Importance of AI in Software Development

As software development becomes more complex and time-sensitive, the role of AI in this field has grown significantly. AI-driven code assistants represent a major shift in how software is developed. They bridge the gap between human expertise and the efficiency of automated processes.

AI in software development not only speeds up the coding process but also enhances the quality of the code. These tools can analyze vast amounts of code, learn from it and apply that learning to assist developers in real-time. They can adapt to different coding styles and requirements, making them versatile tools for a wide range of programming tasks. Furthermore, as software projects increase in size and complexity, AI code assistants become crucial in managing and navigating these complexities. They offer intelligent insights, detect potential issues before they become problematic, and assist in keeping large codebases clean and manageable.

In addition to increasing efficiency, AI code assistants are instrumental in reducing the barrier to entry in software development. They make coding more accessible to people with varying levels of expertise, including those who are new to programming. This accessibility is essential in an era where the demand for software development skills is continuously rising. By providing real-time assistance and learning resources, AI code assistants are not just tools for current developers but also learning aids for future generations of programmers.

10 Key Features to Expect from Code Assistants in 2024

With the rapid advances in large language models (LLMs), the following features are becoming standard in coding assistants as we enter 2024.

Advanced Contextual Understanding

Modern code assistants are evolving to understand the context of code more intricately. Unlike simple autocomplete tools, these AI systems can analyze the broader scope of a project, recognizing patterns and offering suggestions that align with the overall architecture and design patterns. This advanced understanding not only streamlines coding but also aids in maintaining consistency throughout the codebase.

Real-Time Code Optimization Suggestions

AI tools can proactively recommend changes to improve performance and efficiency. For instance, a code assistant might suggest a more efficient algorithm or highlight redundant code blocks that can be simplified. This feature not only enhances code quality but also serves as a learning tool for developers, exposing them to best practices and alternative approaches.

Enhanced Debugging Assistance

Debugging is often a time-consuming aspect of development. In 2024, expect code assistants to revolutionize this process by offering enhanced debugging assistance. Through deep analysis, these tools automatically detect errors and anomalies, suggesting potential fixes. They can pinpoint the exact location of a bug and provide context, significantly reducing the time developers spend in the debugging phase.

Automated Refactoring

Refactoring is crucial for maintaining clean code, and AI-driven assistants now offer sophisticated automated refactoring capabilities. They intelligently suggest changes that improve code readability and maintainability, such as restructuring code or renaming variables for clarity. These suggestions will be particularly beneficial for managing and updating legacy code, ensuring it adheres to modern coding standards.

Code Documentation Generation and Update

Keeping documentation updated is a challenge. Code assistants can now automate this task by generating and updating documentation in real-time. As code changes, the documentation will reflect these updates, ensuring that it always stays in sync with the codebase. This feature will be invaluable in large teams where keeping track of changes can be daunting.

Predictive Coding Assistance

Moving beyond reactive suggestions, code assistants will offer predictive assistance. By analyzing coding patterns and project structures, these tools will predict a developer’s next steps and offer relevant suggestions even before they are explicitly requested. This proactive approach will speed up the development process, reducing the cognitive load on developers.

Understanding and Responding to Developer Queries in Natural Language

Developers can pose queries or express coding requirements in plain English, and the assistant will interpret and execute these requests. Chat-based interfaces offered by leading AI coding assistants take this one step further. This advancement will make coding more intuitive and accessible, especially for beginners or non-expert users.

Real-Time Suggestions and Conflict Resolution in Team Projects

Collaboration in coding projects can lead to conflicts and inconsistencies. Code assistants now offer real-time suggestions and automated conflict resolution for team projects. These tools will identify potential conflicts in merge requests and suggest optimal resolutions, enhancing team collaboration and maintaining code integrity.

Personalized Coding Assistance Based on Past Interactions

Personalization is becoming a significant feature of code assistants. By learning from past interactions with a developer, these tools will tailor their suggestions and support to align with individual coding styles and preferences. This adaptive learning approach ensures that the assistance provided is most relevant and efficient for each developer.

Enhancing Accessibility of Coding for Non-Expert Users

One of the most impactful features of code assistants is their role in democratizing coding. By simplifying complex coding concepts and automating mundane tasks, these tools will make software development more accessible to non-experts. Beginners and those learning new programming languages will find these assistants particularly useful, as they provide guidance and learning aids.

Conclusion

Code assistants powered by AI are becoming an integral part of the software development process. They not only make coding faster and more efficient but also reduce the chances of errors, making the resultant software more reliable. The role of code assistants is only expected to grow, bringing in more advanced and intuitive features. Whether you are a seasoned programmer or a newbie, embracing code assistants can be a game-changer for your coding journey.

February 6, 2024
Does Using AI Assistants Lead to Lower Code Quality?

The emergence of generative AI has permanently altered how code gets written — and due to the massive productivity boost, there’s no going back. According to GitHub, 92% of developers said they use AI tools, and developers complete tasks 55% quicker using Copilot, the popular LLM-based code generator.

Yet, new research has potentially upset the idea that AI assistants like Copilot are positively advancing software development without consequences. The report, conducted by GitClear, analyzed over 150 million lines of code over the last four years and found an upsetting uptick in code churn and a decrease in code reuse since the dawn of LLM-driven software development.

So, this begs the question: Does using AI assistants actually lead to lower code quality? Well, the answer is complicated. “I wouldn’t say that the report proves that AI assistants are reducing code quality since our data is correlational,” said Bill Harding, programmer and CEO at Amplenote & GitClear, the organization behind the aforementioned research. “But I would say that there is a confluence of indicators that show code quality decline at scale, starting around 2022 and accelerating in 2023.”

AI-Generated Code Increases Overall Volume

One potential contributor to lower code quality is the sheer amount of code produced. Since LLMs automate code generation, production is accelerated across the board. Without guardrails, generative AI will likely contribute to more technical debt.

“AI-enabled code development is going to have a major effect on coding, exponentially increasing the volume and velocity of code delivery, much of which we anticipate will be of lower quality and more bloated,” said Mehran Farimani, CEO, RapidFort.

In addition to quality concerns, there are also governance and security concerns to consider from the use of AI and surges of new code production. Some anticipate this will overwhelm security teams, which have already been struggling to keep pace. “Increased developer velocity is going to put security teams under even more pressure,” said Farimani. “The volume of scans will increase, the range of vulnerabilities that need to be addressed will increase and the patches that need to be applied will increase.”

Blindly Relying on AI Outputs

According to GitHub, developers accept code completions from Copilot 30% of the time. This percentage is likely to keep increasing as the LLM is trained on internal style and comfort with the tool increases. However, there is still the risk of developers blindly relying upon generative AI outputs without giving them the proper oversight, which could also affect code quality.

“I think there is a risk for lower quality code if a developer blindly relies on what an AI assistant generates,” said Rob Whiteley, CEO at Coder. “Generative AI tools have flaws, and if outputs are not verified, they can insert code that compiles but does not behave as a developer may have intended. It can also insert errors or bugs if not checked.”

How to Retain Code Quality in the AI Age

“I would not say that results of [the GitClear] study are conclusive,” said Dr. Eirini Kalliamvakou, staff researcher at GitHub. She believes the results are misleading since there is no way to know if an AI or a human authored the code that was analyzed for the study. That said, she acknowledged that “if we are helping people code faster, we should ask the question about code quality.” So, how do we retain the quality of all code, regardless of whether it’s human- or machine-generated?

Use Copilot Thoughtfully

The first recommendation is to use AI assistants thoughtfully. If you’re exhausted at the end of a long day, it’s a lot easier just to “hit tab” and accept an AI-generated auto-complete. But this might not be the best move. “When used thoughtfully and with oversight from the developer, I do think AI can improve the quality of a developer’s code by making suggestions and preventing human errors,” said Whiteley.

Apply the Same Code Reviews to AI-Generated Code

Additionally, development teams should be applying the same degree of rigor to AI-generated code, taking it through the same design review processes and security reviews as you would for any new application. “Just because you’re writing with Copilot doesn’t mean you should give up on code reviews, your linters, looking for security vulnerabilities and all that,” said Kalliamvakou.

Avoid Redundancy When Possible

Leaner code means efficiency boosts and cost-savings. “Much of code is redundant and AI can streamline that redundancy and offer interesting insights, but not without developers ensuring the suggestions the AI tool makes align with the code they are intending to write,” said Whiteley. “Similar to Gmail’s autocomplete sentence feature, AI can complete lines of code with accuracy, but only as long as it accurately guesses where you’re going with your sentence.”

Keep Tabs on the Impact on Quality

Multiple sources cited that around 30% of code is now AI-authored. Harding calls for more efforts to analyze the impact of AI generation on code quality. “It would benefit the greater good to get a clear and data-backed assessment of how quality fluctuates when developers hit ‘tab’ to accept an AI suggestion,” he said. This is a new field of study, and will require new metrics to fully monitor and appreciate the impact AI has on code quality.

Determine How To Measure Code Quality

Similar to the dearth of consensus on what exactly constitutes developer productivity, defining and measuring code quality is just as challenging. Therefore, Kalliamvakou recommends first defining what code quality means to your organization and what the metrics are. For instance, this could be the “number of defects.” Then, the next step would be to set up methods to alert developers about potential quality issues, she said.

The Bottom Line: AI Enhances Productivity…

AI assistants are changing the way code gets made, and much of this is for the better.

Most notably, AI can have a significant impact on enhancing productivity for up-and-coming or inexperienced developers. “If an enterprise makes technology like generative AI broadly available, then it can supplement the work and boost the productivity of the base of the talent pyramid,” said Whiteley. “Put simply, that investment massively impacts the broader organization, not just the top 1% of developers.” Even if such tools make a developer 20% more productive, they can have a truly transformative impact.

While the GitClear study doesn’t prove the culpability of AI assistants, it raises some interesting questions worth exploring, said Harding. Notably, is code becoming more repetitive? If so, that could have some negative outcomes for overall efficiency. “Code generated during 2023 more closely resembles an itinerant contributor, prone to violate the DRY-ness of the repos visited,” the GitClear report said.

… So, Assess Code Quality

“If leveraged correctly, generative AI will not lower the quality of code,” said Trisha Gee, lead developer advocate at Gradle, Inc. “However, it’s important that developers using this technology are critically evaluating AI-generated code rather than taking what AI suggests at face value. Code from generative AI may not always be ‘correct,’ be the best solution to a problem or match the code style of the codebase.”

My takeaway is that, yes, you can retain code quality while using AI assistants. However, this assessment comes with some strings attached. It will require an ongoing effort to sanitize the outputs of AI and instill the same checks we typically apply to manually-generated code.

Image source: https://vecteezy_futuristic-technology-concept-mixed-media-innovations-data_7019122_167-3.jpg

February 6, 2024
Dynatrace Extends Data Reach and Scope of Observability Platform

At its Perform 2024 event today, Dynatrace unfurled a Dynatrace OpenPipeline that makes it possible to apply analytics to multiple types of data sources in real time.

The company also unveiled a Data Observability offering that can be used to vet the quality and lineage of data being exposed to the Davis artificial intelligence (AI) engine at the core of the Dynatrace observability platform to reduce false positives that otherwise might generate an alert in addition to helping reduce the volume of data that might need to be stored.

Finally, Dynatrace announced it is starting today extending the reach of its observability platform to the large language models (LLMs) that are used to create generative AI platforms. That capability will, for example, make it simpler to monitor the consumption of tokens that are used to provide access to these models.

Steve Tack, senior vice president for product for Dynatrace, said Dynatrace OpenPipeline will enable organizations to streamline the collection of data in a way that will enable them to apply observability more broadly by applying stream processing algorithms to petabytes of data.

Scheduled to be available in 90 days, the Dynatrace OpenPipeline capability enables IT teams to ingest and route observability, security, and business events data from any source and in any format at the point of ingestion, including unstructured data that is automatically converted into a usable format. That data can then be enriched to enable deeper analytics.

It also provides IT teams with more control over which data they analyze, store or exclude from analytics, which in turn helps reduce the total cost of observability, noted Tack.

Finally, it presents IT teams with the ability to apply more customizable security and privacy controls to how that data is employed, he added.

Dynatrace is pursuing a multimodal approach to AI that spans predictive, causal and generative models. Collectively, that approach makes it simpler to identify the root cause of issues, identify anomalies that might disrupt services and streamline incident management by, for example, providing summarizations of events in a natural language format.

It’s not clear how broadly observability will be applied beyond the data collected to manage DevOps workflows and other IT processes, but it’s clear there is a high correlation between IT events and business outcomes as organizations become more dependent on software. As AI becomes more pervasively employed, it should become simpler to apply analytics to a much wider range of types of data to surface the relationship between IT events and business processes. In effect, Dynatrace is making it simpler to apply best data engineering practices to collect, manage and analyze that data, noted Tack.

In the meantime, DevOps teams should revisit how their pipelines are currently constructed to make it simpler to capture all relevant data. As critical as telemetry data collected from the DevOps platform is to ensure application development and delivery occurs as fast as possible, that data is only one factor in a larger equation. The challenge and the opportunity now is determining how best to apply AI to correlate it all.

January 31, 2024
Anima Adds Ability to Personalize Generative AI Code

Anima today revealed is has added an ability to personalize the code its generative artificial intelligence (AI) platform creates based on how an application is described using tools from Figma and Adobe.

That capability makes it possible for the code being generated to resemble the coding style of an organization or specific developer more closely.

Anima CEO Avishay Cohen said the company developed this capability to reduce the editing time that is otherwise required when trying to embed code generated by a large language model (LLM) within an application that has a well-defined coding methodology.

The platform that Anima provides is based on a mix of LLMs and heuristics that the company developed to generate code. In this case, the code being generated is written in React or HTML to make it simpler to build an application’s front end. In contrast, a general-purpose LLM such as ChatGPT is not optimized for creating these types of graphical applications.

Developers no longer need to worry about, for example, making sure each pixel is perfectly placed because the Anima platform automates that part of the application development process, including adding responsive behavior, animations and logic, noted Cohen. In addition, customized instructions may be applied to an entire project or even a specific component.

In effect, the platform enables backend developers to automatically create a front-end for their application from a description in tools provided by Figma or Adobe. As a result, developers can allocate more time to writing business logic and optimizing application performance, said Cohen.

Later this year, Anima will integrate its platform with GitHub and other continuous integration/continuous delivery (CI/CD) platforms as part of an effort to extend the reach of the platform across an entire codebase, he added.

There is, of course, already no shortage of generative AI tools for writing code, but the degree to which they have been incorporated into DevOps workflows varies from one organization to the next. Much of the code created using platforms such as ChatGPT can vary widely in terms of quality. These platforms were trained using code collected from across the web so it’s not uncommon for the code generated to include known vulnerabilities that DevOps teams need to scan before incorporating into a build. In other cases, the code generated might simply break when installed.

Regardless of these issues, however, the amount of code being generated by machines is only going to increase, especially as more LLMs that are specifically trained to create code based on examples that have been closely vetted become more widely available.
In the meantime, developers who never especially enjoyed building the visual front ends of applications can now focus more of their time on tasks they prefer. Alternatively, developers who specialize in the front ends of applications will find that much of the toil involved will soon be automated in a way that should provide more time for them to create customized experiences.

One way or another, the overall pace at which applications are built is about to exponentially increase. The challenge now will be finding ways to take advantage of AI to enable DevOps teams to keep pace.

January 30, 2024
SmartBear Acquires Reflect to Gain Generative AI-Based Testing Tool

SmartBear this week revealed it has acquired Reflect, a provider of a no-code testing platform for web applications that leverages generative artificial intelligence to create and execute tests.

Madhup Mishra, senior vice president of product marketing at SmartBear, said the platform Reflect created will initially be incorporated into the company’s existing Test Hub platform before Reflect’s generative AI capabilities are added to other platforms.

Reflect provides access to a natural language interface to create tests using multiple large language models (LLMs) that it is designed to invoke. It can also understand the intent of a test to understand what elements to test regardless of whether, for example, a button has been moved from one part of a user interface to another, said Mishra. Test step definitions, once approved, can also be automatically executed using scripts generated by the platform.

SmartBear has no plans to build its own LLMs, said Mishra. Rather, the company is focusing its efforts on providing the tools and prompt engineering techniques needed to effectively operationalize them, he added.

Reflect is the tenth acquisition SmartBear has made as part of an effort to provide lightweight hubs to address testing, the building of application programming interfaces (APIs) and analysis of application performance and user experience. Last year, the company acquired Stoplight to gain API governance capabilities.

Rather than building a single integrated platform, the company is focused on providing access to lightweight hubs that are simpler to invoke, deploy and maintain, said Mishra. The overall goal is to meet IT teams where they are versus requiring them to adopt any entirely new monolithic platform that requires organizations to rip and replace every tool they already have in place, he said.

There is little doubt at this point that generative AI will have a profound impact on application testing in a way that should ultimately improve the quality of the applications. As the time required to create tests drops, more tests will be run. Today, it’s all too common for tests not to be conducted as thoroughly as they should be simply because either a developer lacked the expertise to create one or, with a delivery deadline looming, they simply ran out of time.

Naturally, the rise of generative AI will also change how testing processes are managed. It’s not clear how far left generative AI will push responsibility for testing applications, but as more tests are created and run, they will need to be integrated into DevOps workflows.

Of course, testing is only one element of a DevOps workflow that is about to be transformed by generative AI. DevOps teams should already be identifying manual tasks that can be automated using generative AI as part of an effort to further automate workflows that, despite commitments to automation, still require too much time to execute and manage. Once identified, DevOps teams can then get a head start on redefining roles and responsibilities as generative AI is increasingly operationalized across those workflows.

January 26, 2024