There are plenty of challenges throughout the software development lifecycle, but one that has proven to be a make-it-or-break-it for projects is requirements management.
Requirements engineering is more than simply listing desired features. It is a comprehensive approach to capturing, organizing and evolving project specifications throughout development. Yet this segment of the development process is often an afterthought.
Poor requirements infiltrate countless projects, and many fail before they even begin. At Crowdbotics, we are intently focused on addressing this issue and devising solutions that create robust requirements, giving organizations the best chance at success. However, solving the requirements problem is only part of the equation; the other is the rapidly evolving development process itself.
This is where modern artificial intelligence (AI) tools are starting to make a game-changing difference.
In just the last few years, tools like GitHub Copilot, Devon, Poolside and others have transformed the way developers approach coding. Yet, while immensely useful, they often operate in isolation. Multi-agent systems — where multiple AI models or tools work in collaboration — have demonstrated significant potential. This led us to ask: What would happen if we brought these advanced tools together in an agentic architecture? Specifically, could we combine our requirements management solutions with top code-generation tools to create something greater than the sum of its parts?
The answer is yes.
What AI Collaboration Looks Like Today
The recent advancement of AI has been staggering, but it is primarily driven by the ‘more data and more compute’ mantra. While this approach has yielded powerful models with emergent capabilities, it becomes increasingly evident that there is a ceiling to what can be achieved by simply scaling these inputs. Chain-of-thought reasoning and retrieval-augmented generation (RAG) have proven valuable, yet they have foreshadowed an even more promising path: Collaboration.
Sharing knowledge and tasks between AI models is taking hold as a way to boost performance without the need for additional data or compute. Despite the potential, however, there has previously been little exploration of how commercially available AI tools could work together as agents in real-world scenarios.
To know more, we conducted an experiment to test how well these tools could collaborate in the DevOps space.
What is important to note here is that while multi-agent configurations have been extensively used in academia and in ‘black box’ systems like OpenAI, there is very little research on commercially available products working together. We believe that this demonstration of commercially available models improving in multi-agent configurations is indicative of what is to come soon.
Experimenting With Multi-Agent Architectures
In our study, we extended our business requirements model to GitHub Copilot, specifically leveraging its code suggestion features. The idea was simple: If Copilot had a more contextual understanding of the business requirements, could it offer better suggestions? To test this, we set up a simple study with three groups:
- Group 1: No AI tooling
- Group 2: GitHub Copilot only
- Group 3: GitHub Copilot + Crowdbotics
Each group then worked on the same roughly four-hour programming task, and we measured the data.
The results were convincing: Those who used GitHub Copilot in tandem with Crowdbotics’ requirements tools saw a 25% increase in task success rates. Additionally, the code suggestion accuracy of GitHub Copilot improved by nearly 14%. This improvement required no additional data, no extra compute and no further training — just the shared context between the two models. These results provided empirical evidence that using commercially available models in a multi-agent configuration can improve performance without additional retraining or increased compute.
AI Unlocks the Power of Collaboration
The study results underscore the untapped potential of AI collaboration. Simply sharing knowledge between two commercially available tools led to significant performance gains. And this was not just an academic exercise — Crowdbotics is available today as a GitHub Copilot extension. Developers can plug it into their integrated development environments (IDEs) to integrate improved PRD context and experience the benefits firsthand.
Perhaps even more importantly, the findings point to a larger truth: The future of AI is collaborative. The siloed nature of many current AI tools and systems limits the industry, however, by fostering interoperability and multi-agent systems, we unlock new efficiencies and capabilities that were previously out of reach.
I am personally excited about this vision. The path to a better and more productive future involves more models working together, combining their strengths to overcome individual limitations. As AI evolves, we are committed to exploring and advancing this collaborative paradigm, building tools that enable a more connected, efficient and innovative software development ecosystem.
Ultimately, the journey has just begun, but the potential for a brighter future for AI and DevOps is already evident.