devopsy.kinsta.cloud

Category: Continuous Testing

Continuous testing is the process of executing automated tests as part of the software delivery pipeline to obtain immediate feedback on the business risks associated with a software release candidate.

  • Coexisting With AI: The Future of Software Testing

    Coexisting With AI: The Future of Software Testing

    If 2023 was the year of artificial intelligence (AI), then 2024 is going to be the year of human coexistence with the technology. Since the release of Open AI’s ChatGPT in November 2022, there has been a steady stream of competing large language models (LLMs) and integrated applications for specific tasks, including content, image processing and code production. It’s no longer a question of if AI will be adopted; we have moved on to the question of how best to bring this technology into our daily lives. These are my predictions for the software quality assurance testing industry for 2024.

    Automated testing will become a necessity, not a choice.

    Developers will lean heavily on AI-powered copilot tools, producing more code faster. That means huge increases in the risk profile of every software release. In 2024, testers will respond by embracing AI-powered testing tools to keep up with developers using AI-powered tools and not become the bottleneck in the software development life cycle (SDLC).

    The role of the tester will increase and evolve.

    While AI is helping software engineers and test automation engineers produce more code faster, it still requires the highly skilled eye of an experienced engineer to determine how good and usable the code or test is. In 2024, there will be a high demand for skilled workers with specific domain knowledge who can parse through the AI-generated output and determine if it’s coherent and useful within the specific business application. Although this is necessary for developers and testers to start trusting what the AI generates, they should be wary of spending inefficient amounts of time constructing AI prompts, as this can ultimately lead to decreased levels of performance. For instance, a developer could easily spend most of their time validating the AI-generated output instead of testing the release that will be deployed to users.

    Being able to distinguish between when to rely on AI and when to forego AI’s help will be key to streamlining the workflow. Eventually, we’re going to start seeing AI-powered testing tools for non-coders that focus on achieving repeatability, dependability and scalability so that testers can truly use AI as their primary testing tool and ultimately boost their productivity.

    The rise of protected, offline LLMs and the manual tester.

    As enterprise companies show signs they don’t trust public LLMs (e.g., ChatGPT, Bard, etc.) with their data and intellectual property (IP), they’re starting to build and deploy their own private LLMs behind secured firewalls. Fine-tuning those LLMs with domain-specific data (e.g., banking, health care, etc.) will require a great volume of testing. This promises a resurgence in the role of the manual tester as they will have an increasingly important role to play in that process since they possess deep domain knowledge that is increasingly scarce across enterprises.

    As we stand on the brink of 2024, it is evident that the synergy between artificial intelligence and human expertise will be the cornerstone of software quality engineering. Human testers must learn to harness the power of AI while contributing the irreplaceable nuance of human judgment. The year ahead promises to be one where human ingenuity collaborates with AI’s efficiency to ensure that the software we rely on is not only functional but also reliable and secure. There will likely be a concerted effort to refine these collaborations, ensuring that AI serves as a reliable support rather than an overpromised solution.

  • 2024: The Year of Testing

    2024: The Year of Testing

    Caveat: I currently cover DevOps – including test – and security – including the *AST. This does give me a viewpoint that others may not have, and also potentially gives me blinders that others may not have. I am not generally a predictor but a reader of direction. Proclamations like this are rare for me and are shorter-term/more tactical than the grab bag of predictions usually seen this time of year.

    I was thinking about this one this morning; the direction of travel of traditional and security testing is very similar at the moment. We are seeing the vast complexity of the overall space and the rapid growth of automation come together to once again make all testing a topic worthy of consideration.

    I loved test-driven development (TDD). I can’t speak for others, but RPC/API is just a cool way to implement centralized functionality, and TDD is almost mandatory to develop APIs and the clients that call them. There is no UI to distract or have separate requirements in API development; there is a stated function and correct or incorrect behavior. To a computer scientist, this is predictable beauty. TDD made it more than predictable—it made it a trail through the woods. “We know it needs to do X, Y and Z, with the only side effects being A,” can be detailed with tests to prove that is what it does, and then look to prove the “It doesn’t have other side effects” part. And you can do it all while developing—no need for dev/test iterations. Like any IT solution, it isn’t a great fit for everything, but even if it isn’t called TDD, it is nearly impossible to develop API-based solutions without some level of TDD.

    The thing is, TDD is absolutely a developer’s tool. Developers and DevOps pretty much must use it to turn out usable APIs. The test team only has a use for a subset of those tests. Because if the dev team did it right, they passed all the TDD tests, and the test team’s role in that scenario is to look closely at what the developers might have missed. Integration is often beyond the control of an individual dev and/or API, for example, because many different clients may be integrating with it. Test teams, in a true TDD shop, fill the gaps with meaningful tests that evaluate the overall application quality.

    Security testing—at least static application security testing (SAST) and increasingly dynamic application security testing (DAST)—are also becoming a core part of the development process. This is as it always should have been. It is far easier to fix a security flaw (or any flaw) at the design/early code stage than in a testing process after deployment. The quality of security testing inputs is also increasing, making it far more viable. And finally, the tools are being effectively integrated. Being a developer who did a lot of security work because we often needed it and had no one to do it, I will say I get a little rush about being told right in my IDE about the security risks in my code. I write, I see, I revise where necessary, and I turn out better code. All without having to pause for security or bang heads over policy.

    To date, all this testing is separate, but I suspect, having vaguely the same location in the process of development, that testing will all occur together and the markets may even start to merge a bit—though merger of security testing and functional/performance testing tools is a bit of a stretch, it is more possible than in the past. In most orgs, even though developers have an increasing stake in both sets of testing, the target purchasers are still two different groups—security versus test—with different priorities. That’s why I think they aren’t likely to merge any time soon.

    While AI in most technology spaces is currently experiencing the same level of FUD as XML, Java, JavaScript or low-code was in the past, all super-hyped tools in tech do have a sweet spot. Security and testing are in that sweet spot for AI. Both in test generation and in results evaluation, it does it better, faster and with more coverage while giving staff a concrete set of results to pore over and improve security posture and/or code quality. I cannot say loudly enough that in terms of enabling the steps in the SDLC that have suffered from a lack of resources and timeline to write, execute and evaluate tests, AI is a game-changer.

    So, all of these things together mean that 2024 is a good time to take a hard look at your testing environment(s) and architecture, both traditional testing and security testing. Now that AI has made advanced automation a fact, it is time to consider implementing the level of testing we always knew we should have but never had the resources. Build it right into the build process and make it part of the organizational DNA—just like Agile is.

    If you could hire a specialist who would paw through your code and give you a list of the 25 things you need to fix first, you would do it. That is, essentially, what modern testing tools are. They can do more tests, evaluate more issues and side effects and filter down to the “No—this is really important!” list without costing the team a ton of time.

    And keep rocking it. We’re all sitting on bugs and vulnerabilities. I do hope we are way past denying that fact. Apps are big old complex monsters; these days, sometimes built from generations of technologies cobbled together. Of course they have issues. Even yours. So let’s take steps to clean it up, just a little bit. And keep clipping along.

  • Synopsys Report Sees Steady Application Security Gains

    Synopsys Report Sees Steady Application Security Gains

    An analysis of the security automation practices of 130 organizations published today by Synopsys suggests significant progress has been made toward securing software supply chains in the last year. However, much work still remains to be done.

    The annual Building Security In Maturity Model (BSIMM) report found there has been a 22% increase in the number of organizations creating software bills of materials (SBOMs) this year. Additionally, there has been a 10% increase in the number of organizations tracking open source software risks.

    At the same time, there has been a 25% increase in the number of organizations that are fixing all defects found in software and a 15% increase in the number of organizations that can identify potential attackers and track attack patterns.

    The report also sees significant gains in activities related to software development practices (44%), penetration testing (35%) and compliance and policy controls (21%). Usage of application behavior monitoring and diagnostics grew 64% and monitoring automated asset creation grew 45%, respectively.

    The finding and publishing of secure design patterns, the required use of approved security features and frameworks and the use of application containers to support security goals all grew about 25%, according to the report.

    However, the report also noted that usage of possible attack lists dropped by 31%, while expert-driven tasks such as building and applying adversarial security tests declined 25% and use of centralized defect reporting dropped 18%.

    The most widely used processes are implementing security checkpoints and associated governance (91%), creating or interfacing with incident response (90%), identifying privacy obligations (88%), using external penetration testers to find problems (88%), ensuring host and network security basics are in place (87%) and automated code review tools (86%).

    Jamie Boote, associate principal security consultant for the Synopsys Software Integrity Group, said the report suggests more responsibility for cybersecurity is shifting everywhere as organizations rely more on automation to apply security controls. Application development and IT operations are, as a result, able to address vulnerabilities more proactively either before an application is deployed or as soon as a vulnerability is discovered, he noted.

    The challenge organizations will encounter when it comes to automating security processes is there is a tendency to set and forget. Once a process becomes automated, not enough organizations will reevaluate that process as the cybersecurity threat landscape continues to evolve, noted Boote.

    Of course, the rate at which security processes are being automated varies widely from one organization to the next, but the more automated they become, the less dependent organizations become on application developers who typically don’t have a lot cybersecurity expertise. All too often, application developers are rushing to meet software delivery deadlines at the expense of security best practices. Unfortunately, it’s usually only a matter of time before an issue that was avoided is later discovered in a production environment and is considerably more problematic to update, given all the dependencies between software and infrastructure that exist.

    One way or another, however, a pending wave of regulations will soon force organizations to pay more attention to securing software supply chains. The only thing left to determine in the absence of automation is how painful the reengineering of DevOps workflows to embrace DevSecOps best practices is going to be.

  • Google Taps Nobl9 to Improve Services Reliability

    Google Taps Nobl9 to Improve Services Reliability

    Google announced this week it has added a capability for tracking reliability based on how service level objectives (SLOs) are being met that is based on a platform developed by Nobl9.

    Nobl9 launched a Nobl9 Reliability Center earlier this year that provides a single source of truth about the state of reliability in an application environment. Included in that platform is SLOgpt.ai, a generative artificial intelligence (AI) tool for programmatically creating SLOs and launching natural language queries to determine how well they are being adhered to across a distributed computing environment. The generative AI tool developed by Nobl9 is based on Vertex, a foundational large language model (LLM) provided by Google.

    Kit Merker, chief growth officer for Nobl9, said the generative AI tool also makes managing SLOs more accessible to a wider range of stakeholders.

    The expansion of the alliance with Google is the latest in a series that will make it simpler to manage SLOs across multiple ecosystems, including, most recently, partnerships that have been established with Microsoft and Cisco, noted Merker.

    The Nobl9 Reliability Center is already integrated with more than 50 DevOps, observability and incident management tools. It collects metrics, events, logs, traces and alerts to track incidents, releases, rollbacks, runbooks and other documentation. Those integrations will make it simpler for IT teams to create SLOs using historical data that would otherwise be challenging to define from scratch.

    In time, there is no doubt tools for managing SLOs will become more widely available, but given the dependencies that exist between applications and systems, DevOps teams will need an approach that can be employed across hybrid IT environments, said Merker.

    In general, as more microservices-based distributed applications are built and deployed, it’s becoming more challenging to maintain SLOs across them since they have many more dependencies than legacy monolithic applications. Each feature and application programming interface (API) added over time can adversely impact performance and reliability. The challenge is that it’s not so much that any given service will outright fail as much as it is determining where degradations are occurring that result in SLOs not being met.

    Each IT team needs to provide some sort of objective benchmark that assesses their overall effectiveness at delivering application services by making it simpler to track whether SLOs are achieved and maintained. Not every service needs to necessarily meet a stringent requirement, but IT teams do need to have confidence in the fact that, on average, certain levels of service are being consistently maintained.

    When it comes to SLOs, the challenge is first determining which dependencies might impact the level of reliability that can be maintained in the event of an outage and then, second, making sure whenever necessary workarounds—such as rerouting application programming interface (API) calls to other services—can be easily invoked.

    SLOs, of course, as a concept, have been around for decades, but with the rise of digital services that span multiple organizations, the level of reliability DevOps teams are expected to provide is now being written into business contracts. As a result, the overall level of stress related to managing SLOs is only going to increase in an era where businesses have never been more dependent on software.

  • BMC Adds More Tools to Modernize Mainframe Applications

    BMC Adds More Tools to Modernize Mainframe Applications

    BMC this week added a BMC AMI DevX Code Insights tool that makes use of machine learning algorithms to break apart monolithic COBOL mainframe applications into subprograms that are easier to update and refactor.

    In addition, BMC is now making available BMC AMI zAdviser Enterprise, a commercial instance of a previously announced free tool that uses machine learning algorithms to collect telemetry data from mainframe applications. The commercial version of the tool adds a dashboard built and maintained by BMC that makes it simpler to consume that data.

    John McKenny, senior vice president and general manager of Intelligent Z Optimization and Transformation at BMC, said BMC AMI DevX Code Insights visually maps a mainframe application to analyze data flow and debug applications made up of millions of lines of code in real-time that would otherwise be too complex to unravel. Armed with the insights gained, IT teams can then isolate business logic, refactor subprograms, add application programming interfaces (APIs) or remove dead code as they see fit, he added.

    BMC AMI DevX Code Insights also makes it simpler for organizations to onboard new developers to a project at a time when many of the original developers of COBOL applications are retiring, said McKenny.

    In the coming year, BMC is also committed to adding generative artificial intelligence (AI) tools to its portfolio for reengineering mainframe applications, added McKenny. Those tools, for example, will make it simpler to create documentation for legacy applications, he noted.

    It’s not clear how quickly organizations that have mainframes are looking to re-engineer applications, but as more of them adopt DevOps best practices, they are being updated more frequently. The less monolithic the application, the simpler that task becomes. The issue is that mainframes tend to run large applications that access large amounts of data, so many IT teams are naturally fearful of breaking them if they are reengineered, noted McKenny.

    While there is no doubt that the process of reengineering applications will become more automated in the years ahead, software engineers will still be needed to manage the overall process, said McKenny. The role of the software engineer will undoubtedly evolve, but it’s not likely software development and deployment will be wholly automated any time soon, he noted.

    In the meantime, IT organizations that manage mainframes continue to embrace DevOps best practices to varying degrees. There may be some applications that they continue to build and deploy using waterfall methodologies, but as mainframe applications become integrated into digital business workflows, the need to update them faster becomes more apparent.

    The challenge, as always, is not only providing IT teams with the tools to make that transition but also navigating the cultural issues that inevitably arise any time a change of this magnitude is made to the software development life cycle (SDLC). Regardless of the approach, the building and maintenance of mainframe applications are evolving. The only issue that remains to be resolved is finding ways to reduce the level of friction that transition creates as quickly as possible.

  • CircleCI Extends CI/CD Reach to AI Models

    CircleCI this week revealed it is extending the reach of its continuous integration/continuous delivery (CI/CD) platform to make it simpler to incorporate artificial intelligence (AI) models into DevOps workflows.

    In addition to providing access to the latest generation of graphical processor units (GPUs) from NVIDIA via the Amazon Web Services (AWS) cloud, Circle CI has added inbound webhooks to access AI model curation services from providers such as Hugging Face and integrations with LangSmith, a debugging tool for generative AI applications and the Amazon SageMaker service for building AI applications.

    CircleCI CEO Jim Rose said while there is clearly a lot of enthusiasm for incorporating AI models into applications, the processes being used are still immature, especially in terms of automating workflows that include testing of probabilistic AI models.

    Most AI models are built by small teams of data scientists that create a software artifact that needs to be integrated within a DevOps workflow just like any other artifact, noted Rose. The challenge is that most data science teams have not yet defined a set of workflows for automating the delivery of those artifacts as part of a larger DevOps workflow, he added.

    DevOps teams will also need to make adjustments to a version control-centric approach to managing applications to trigger pipelines to pull AI software artifacts that exist outside of traditional software repositories. For example, the inbound webhooks provided by CircleCI now make it possible to automatically create a pipeline whenever an AI model residing on Hugging Face changes.

    It’s still early days as far as the deployment of AI models in production environments is concerned, but there is no doubt generative AI will have a major impact on how software is developed. AI models are a different class of software artifacts that are retrained instead of being updated, a process that occurs intermittently. As such, DevOps teams need to keep track of each time an AI model is being retrained to ensure applications are updated.

    At the same time, generative AI will also increase the pace at which other software artifacts are being created and deployed. Many of the manual tasks that today slow down the rate at which applications are built and deployed will be eliminated. That doesn’t mean there will be no need for software engineers, but it does mean the role they play in developing and deploying software is about to rapidly evolve. DevOps teams need to evaluate both how generative AI will impact the tasks they manage as well as the way the overall software development life cycle (SDLC) process needs to evolve.

    Each organization, as always, will need to decide for itself how best to achieve those goals depending on the use cases for AI,  but the changes that generative AI will bring about are now all but inevitable. The longer it takes to adjust, the harder it will become to overcome the cultural and technical challenges that will be encountered along the way.

  • KubeCon 2023: Internal Developer Platforms and Developer Productivity

     

    Mitch Ashley: Hey there, everybody. Welcome back to Chicago to KubeCon 2023. We are on day two of our live-streaming on Techstrong TV across all of our sites, devopsy.kinsta.cloud, Security Boulevard, Techstrong.TV, and Cloud Native Now. So, we’ve got more great interviews. Alan’s just finished a couple and I’m teed up now with Amit Gorvin. Nice to meet you.

    Amit Eyal: Very nice to meet you.

    Mitch Ashley: And you are with, say the name.

    Amit Eyal: Kubiya AI.

    Mitch Ashley: Kubiya AI, okay. That’s a lot of syllables there, but okay. I’m just kidding you.

    Amit Eyal: We do this on purpose to confuse.

    Mitch Ashley: You know what? Once you hear it, it’s unique, right? You remember those [inaudible 00:00:45].

    Amit Eyal: Hundred percent.

    Mitch Ashley: Exactly. Okay, so I know you’ve talked with this before and you’ve talked about IDPs. It’s a really fast-growing area, right? Everybody’s focused on-

    Amit Eyal: Absolutely.

    Mitch Ashley: How do we help developers, platform engineers, the people all working together to make lives easier, self-service, that kind of thing, portals and all of that. I know you had some announcements recently. Talk about that, but maybe set it up of what you all do and why that’s a little different.

    Amit Eyal: Sure. I think maybe to take one step back, one of the things that I think a lot of companies in the industry, including ourselves are doing is, how do you create a better exchange between the developers and the DevOps, the operators on one side, developers on another hand where you have the golden path towards productivity. IDPs, internal developer platforms, have been known backstage being the primary leader in this space to unlock lab productivity.

    One of the things that we, at Kubiya, have discovered, and this is what we’ve tried to essentially address, is the self-service is only as good as the interaction between the users and the tools. And where often it does break down is on the fact that it’s a one directional exchange between the user and the tool. If you fail on authentication, fail on permission, don’t know the variables you need to pass or anything just doesn’t work as expected, you have a question, you have to go back and do a Jira ticket queue in order to go and to essentially get assistance from a human being.

    So, what has come to fruition in the last year, obviously ChatGPT doing the market education for this, is conversational AI, having a two directional exchange between machines and humans. We’ve married these two worlds together. We’ve created that exchange between your users, developers, and operators with the tools in a way where you can have a full-on conversation with it.

    You can go and spin up a Kubernetes deployment, it will ask you how many you replicated. You can scale it up, scale it down. You can have for your Jira ticket queue, for example, and ask it questions about who owns what, where they’re assigned creation, notating, anything that you would be able to do as a human, only in this case, you’re just having a conversation. Because imagine having on the other side, someone like Fred, John, Bob on the on-call channel having a conversation with you. We’re replacing that with the virtual assistant.

    Mitch Ashley: Interesting. So, replacing the virtual assistant model. It sounds like also you were maybe describing, if I got this right, that the developer technical staff can interface through ChatGPT to say… I mean the nice thing about ChatGPT is you can talk to it like a programming language just in conversational logic, right? I want you to create this for me with blah, blah, blah. Is that part of what you’re saying? You can give it instructions or requests?

    Amit Eyal: One minor information, we’re not using ChatGPT. It’s a ChatGPT-like experience.

    Mitch Ashley: I see.

    Amit Eyal: It’s probably the best way to anchor people to-

    Mitch Ashley: I jumped that.

    Amit Eyal: Which is fine. We’re using about eight different language models, both large and small. We’re actually using conversational AI piece with GPT-4 for Azure, just a FYI, but that’s separate. The experience is ChatGPT-like. Only here, it’s your tools, your programs, your scripts, essentially a personalized IDP, if you may, for each and every persona in organization, each and every individual in the organization.

    Mitch Ashley: Okay. So, are you using generative AI large language models underneath, or that’s just the experience that you’re delivering with whatever technology?

    Amit Eyal: Certainly. So, large language models are giving that natural exchange, essentially the benefits of large language models and the generative piece we’re using. But we’re also harnessing and scaffolding it with rule-based systems, with sound engineering practices. So, that’s how you get the benefit of an expected outcome on one hand. Our users would expect an expected outcome. Essentially, it has to be deterministic what they’d get on the other end.

    Mitch Ashley: I’m glad you said that because I was thinking down the ChatGPT line. It’s nice, but you don’t get the same response, same outcome. It’s not necessarily guaranteed to be consistent, to be accurate. So, it makes sense on the backend scaffolding part.

    Amit Eyal: And there’s permissioning and rule-based systems in place to harness the LLM to function the way you would expect it to.

    Mitch Ashley: Very nice. So, you talked about the developer experience, and I totally get the cognitive dissonance between I’m working in my IDE and doing my developer thing, and then I got to go to this ticketing system or go to some weird operating portal that wasn’t built for developers, but I still got to use it in a huge sense.

    Amit Eyal: Maybe it is, but it’s only as good as how it’s been upkept, maintained or what you’ve actually input into there. So, you still have to go in to program it to act the way you expect it to.

    Mitch Ashley: And it’s part of the flow too, right?

    Amit Eyal: Yeah.

    Mitch Ashley: I might not know answers to all 10 questions at the start of the interaction, but I need to get it started, and then finish it at some point in the process. But it’s about in the flow of the developer work, not I’m a form, fill me out. Right?

    Amit Eyal: A hundred percent.

    Mitch Ashley: Great. So, the announcement then, summarize again for us because I stepped on it with the GPT stuff.

    Amit Eyal: Sure, yeah. No, it’s fine. So, we came out last year actually with our initial platform, which was more resembling next generation of workflow automation because it was more so to get people connected to the world of conversational AI into the world of IDPs. Where we took it from there, and especially with the evolution of large language models, is an agent-based approach where we’re using different models and different agents trained on different models in order to map out effectively different functions within a transaction, almost like the human brain and the lobes interacting with each other.

    So, when you go and interact with the system, depending on how you approach it, it could be, show me how I do something. Say, set up a VPN. That may be in your docs. So, we’re using RAG techniques obviously to go into a train on your knowledge base. But if you’re going to go and run a query against a data source or a database, or maybe show me all my Jira tickets for current sprint for certain… That’s a different language model that’s interacting with you.

    And then if you’re going to go and create a Terraform, create a Kubernetes deployment, destroy Terraform, that’s an action base that’s also needing to go into verify permissions. We’re using open policy agent. We’re also using all the techniques to or data masking, and then we obviously have the orchestration there, a state machine that will go in to orchestrate all of those with the business logic of the organization. So, there’s a lot going on behind the scenes.

    Mitch Ashley: There is. Well, a lot of different kinds of users’ requests.

    Amit Eyal: To the user, it’s seamless. You have a brain emoji thinking and then reading back to you what it’s doing. So, it’s actually very fun interaction, human-like interaction, but then behind the scenes, there’s a lot that’s going on.

    Mitch Ashley: Very cool. Do you integrate at all with IDEs and go that far with it or is it a separate portal?

    Amit Eyal: Users can integrate with their IDEs. They can go and plug in. We have a SDK and it’s all open API based. So, essentially, you can connect to any tool.

    Mitch Ashley: I think it’s really nice and important because for one, it’s right there handy. And also, if you’re delivering any information that you can deliver in the way the code looks like or structured or auto-fill, you can mimic the behaviors of what you do in an IDE that… Very natural. So, I don’t have to go look up an API or what that call is; it’s right there. Or I don’t have to go look up what that thing’s called on my request; it’s right there.

    Amit Eyal: Agreed. So, I think maybe the one call-out I would love to make is people are thinking about the co-pilot approach or ChatGPT approach as a code completion tool. What we’re doing is, we’re doing an operations’ completion tool. Essentially, all of the next step, what would you do, how would you deploy the code, run your CI/CDs and maintain it for day two operations, so that’s where you can think about us filling that gap.

    Mitch Ashley: Okay, interesting. So, congratulations on the announcement.

    Amit Eyal: Thank you.

    Mitch Ashley: That’s fantastic. And every time you come, you’ve got new, in progress and things happening. I’m curious, maybe you’ve talked to Alan about this or Mike, your backstory, you’re CEO, founder of the company. How did you get going down this path? Was that, I was doing this at a big company and this is like a huge need, I’m going to go forward or?

    Amit Eyal: It’s a great question. So, the first time we came on it was myself and my co-founder, who he himself is an operator who built a self-service platform for his platform team in his previous company at Bluevine, and I came from AWS where I was managing partnerships for all the DevOps and DevSecOps-

    Mitch Ashley: Nice couple.

    Amit Eyal: Tier one players. So, I saw where they were possibly struggling with some of this self-service, and where I was in the middle of trying to assist them. And that’s where the thesis on the business side, which is this is a much larger problem and how we’ve actually solved this ourselves came together.

    Mitch Ashley: Interesting. Talk a little bit about your experience in the partnership side of it. Has that come to fruition as an important part of the business? I would imagine when you’re integrating with lots of things, just to look up the API, you can do it. You want to build relationships, partnerships with other companies.

    Amit Eyal: So, we have a very strong ecosystem of partners really looking to have the conversation. I experienced essentially that assistant experience on top of their platform. And right now, we’re letting into queue a few of the strategic ones that we feel that we can both have a strong alignment with, but also maintain. As we grow our team, we’re going to have a much larger ecosystem. We’re already at probably 10, 15 very strong players in the industry who want to have a partnership in place. So, I feel it’s only going to grow from that point.

    Mitch Ashley: Good, good. Amit, you already know this. I learned that lesson on one of the products I worked on where you didn’t design it with you were going to have a partner ecosystem in mind. You get to a point like, okay, we can’t do what we want to do. We got to go back and add that in. So, having that perspective upfront, if that’s a place you’re going to want to go at some point in the life cycle, it’s-

    Amit Eyal: That’s exactly it. Our ecosystem is built to be two-way conversation with all tools. So, if our customers are asking for it, there’s no reason why the vendor on the other side wouldn’t have a stronger alignment so we can have a better user experience for both sides.

    Mitch Ashley: I’m curious, do you sell to the engineering development community? Do you sell more to the platform engineering, operational DevOps tool folks? Where do you tend to focus?

    Amit Eyal: I would say usually that decision is made jointly. The platform teams typically have to have a handshake with the engineering managers. So, whether it’s an engineering manager taking us to the platform team, platform team taking us to the engineering manager, typically, we see both sides involved at least in validating this. But the easiest way to get started is really with the platform teams because they’re the power users typically of the system. They’re the ones creating and scaffolding the configurations of these systems, so it’s easiest for them to get started, test it internally on themselves. And very easily, we usually see that they roll it out to the rest of the engineering team.

    Mitch Ashley: They already have a good idea of what kind of requests come their way.

    Amit Eyal: Precisely.

    Mitch Ashley: And it would make that easier for people.

    Amit Eyal: Otherwise, the other way around would be, let me go and bring in a platform team. So, it’s a more direct approach [inaudible 00:13:09].

    Mitch Ashley: That makes sense and it’s nice. I could totally see. It’s starting, “Well, hey, something might make it easier to take a load off-

    Amit Eyal: Precisely.

    Mitch Ashley: The platform engineering team” or them saying, “Hey-

    Amit Eyal: We’re here to augment them.

    Mitch Ashley: … that’s an easier way to work with this.” Right?

    Amit Eyal: We’re here to augment them. Not everybody has a luxury of hiring three to five more headcounts for next year, but if you could do so with one or two headcounts and then Kubiya, now you can scale.

    Mitch Ashley: Great. If you look forward a little bit, maybe this next six months or maybe 12 months, six months or so, what things are happening in the industry, maybe where you’re heading, what are you thinking about next?

    Amit Eyal: The most recent announcements by OpenAI have been very interesting, because from our perspective, it’s giving us even more tools in order to get to our end goal, which is, in my opinion, a self-driving operator. A self-driving operator that will be able to take end-to-end tasks away from human beings and really free them up to do the highest and best use of their time. It’s still going to require human-in-the-loop interaction and we don’t expect it to be overnight. But I think a lot of the things that was announced recently is already in line with what we’ve been doing, but it’s only making it easier and again unlock more and more use cases.

    Mitch Ashley: Seems like OpenAI have done a pretty decent job of, don’t throw it all out there, incrementally add capability when you’re ready to handle it.

    Amit Eyal: Yes. Because we’ve been in this space a couple of years, we’ve been thinking about, we’ve already been architecting our platform to be conversational native. That’s allowing us to move very fast in this space.

    Mitch Ashley: Great. Well, tell folks where can they go check it out?

    Amit Eyal: Absolutely.

    Mitch Ashley: Can they get a demo online or what can we do to help?

    Amit Eyal: That’s a beautiful part. We open our platform up for self-service. So, you could actually sign up at Kubiya.ai, www.kubiya.ai. Sign up for a free trial. We also have a sandbox where you could, without needing to put in organizations credentials, anything. It’s a gated experience. Obviously, it’s a bit more limited, but you could go and have that nice exchange with Kubi, our DevOps assistant, and really enjoy the experience before committing to going into a trial.

    Mitch Ashley: Appreciate the sandbox. You don’t want to quite turn it loose on your infrastructure quite yet. Let’s kick the tires and then get there.

    Amit Eyal: Mostly read only. We have a few write actions in there, but typically, we want to make sure it’s rated PG for what people can do.

    Mitch Ashley: Exactly. Exactly. Great. And it’s spelled K-U-B-I-Y-A.ai.

    Amit Eyal: Y-A.ai.

    Mitch Ashley: Great.

    Amit Eyal: Thank you very much.

    Mitch Ashley: It was on the… I’m sure on your title but-

    Amit Eyal: Yes, it would be there.

    Mitch Ashley: Amit, pleasure talking with you and congrats on the new progress, new release and capabilities, where you’re headed next. Great market to be in.

    Amit Eyal: I appreciate it and I appreciate your time here today.

    Mitch Ashley: Of course. Great to have you on. For our folks, for the people in the audience, Amit’s representative of some of the great things that are happening not just at KubeCon but in our industry, and progress that’s happening overall. So, stay tuned. We’ve got more great interviews coming back up. Don’t go away. We’ll be right here in a few minutes. So, we’ll be back.