devopsy.kinsta.cloud

Category: DevOps Toolbox

  • Clubhouse Becomes Shortcut to Transform Software Project Management

    Clubhouse Becomes Shortcut to Transform Software Project Management

    Shortcut, formerly known as Clubhouse Software, today announced that its project management software for software development teams has been expanded to include additional workflow capabilities.

    CEO Kurt Schrader said team-to-workflow functionality is being added to the software-as-a-service (SaaS) platform and will make it easier to share workflows across software development teams in a more standardized way.

    Organizations can standardize workflows across multiple teams while still giving each team the ability to customize workflows for their own unique processes. As work gets created and assigned, it automatically moves into a workflow without having to manually update processes.

    Shortcut is designed to make it simpler for development teams to collaborate in a way that maintains transparency with other teams working on projects that are part of the same larger initiative, said Schrader. It allows product managers, engineers and designers to plan, track and measure their work without friction as the organization continues to grow in size, he added.

    For example, Shortcut will surface what teams need to focus on as priorities and timetables change within a specific project in addition to allowing users to define sprints using start and end dates, and the tool then sends reminders so the team can reserve time to focus on specific projects.

    Shortcut will also integrate with continuous integration/continuous delivery (CI/CD) platforms via application programming interfaces (APIs) to enable the development of custom workflows that include, for example, integration with real-time conversations occurring via the Slack collaboration service.

    Most of the project management software DevOps teams employ today is too complex and intimidating for the average business executive. Rather than forcing business users to learn how to master these applications, a case can be made for a more intuitive set of project management tools for both software development teams and the product managers they ultimately serve.

    As more organizations realize how dependent they are on software to drive business processes, many of them are reviewing the tools they currently rely on to manage software development. The goal is to not only reduce friction and enable more transparency across teams but also to provide stakeholders outside of the development team with more visibility into the status of any given project.

    At the same time, organizations are simultaneously launching more software development projects that have many dependencies. Many of those initiatives are based on microservices that are being built in isolation from one another before being aggregated to create an application. The dependencies between those microservices require organizations to make sure they are developed and deployed in a way that doesn’t leave one software development team waiting on another to finish a project before they can complete their work.

    Shortcut

    Schrader said that humans, as a species, are bad at organization. Project management applications should make it simpler for companies to understand how projects are not only organized but also all the dependencies that exist between them.

    Regardless of how software development projects are managed the need for greater visibility is obvious to all. Less clear is the best way to go about achieving that given all the communications channels available.

  • 5 App Modernization Best Practices

    5 App Modernization Best Practices

    An organization’s digital transformation is ineffectual if it doesn’t bring out the best from its applications. This is a near-universal concern since most enterprises still have core applications that are architected primarily to work in legacy IT landscapes in an on-premises dedicated environment, slowing down IT and business performance, stymieing innovation and increasing vulnerability to risk. Businesses are paying the price of their technical debt by forsaking agility, adaptability and competitiveness. That’s bad enough in regular times; in a post-pandemic scenario, this can have devastating consequences. 

    Acquiring a few cloud-native applications that are not integrated with existing systems is of limited value and, at best, provides just part of the answer. On the other hand, overhauling the legacy application landscape is usually out of the question. The only reasonable solution, then, is application modernization. But where does an enterprise begin?

    Pick and Shift

    A great place to start is by examining existing applications. Understanding every application’s function, performance and issues and what it can gain from modernization will help to establish a ‘universal set’ to pick from. The candidates for modernization can be prioritized based on technology factors—for example, when the architecture adds cost, vulnerability or complexity—as well as business considerations, such as compromised agility or value potential. Finally, the enterprise needs to weigh the potential benefits, especially ROI, against the difficulty of modernization.  

    Typically, the biggest, most important applications at the core of the enterprise are the hardest to modernize. But without their involvement, the transformation will most often be incomplete. Rather than lifting and shifting these monolithic applications and their vast data to the cloud, the enterprise can enclose them in an API layer to connect them to all the other applications that are being migrated as well as any new applications that are deployed. Using APIs to expose the functions and data of a legacy core system, such as ERP, allows it to be widely used, reused and modified, as well as leveraged to build new solutions and services. 

    Split Into Microservices

    Refactoring is a useful way of modernizing legacy applications that involves breaking them into microservices, which are then connected to cloud infrastructure such as Docker or Kubernetes. These microservices, which are loosely coupled, can be ‘called’ and scaled individually. Since refactoring requires significant rewriting and restructuring of the code, it is best done progressively over several iterations to make sure everything is functioning properly.  

    Provide Full-Stack Visibility 

    Even when an application is modernized into microservices on the cloud, it needs to communicate with other legacy systems in the enterprise. The problem is that now there are more services and environments to monitor and secure, not to mention single points of failure, which not only increases complexity but also denies the enterprise full-stack visibility. But visibility across environments—edge to core (on-premises) to cloud—is imperative for success. This is achieved by ensuring that the modernized applications fit within the organization’s data fabric. The enterprise needs to capture data across the stack, from end to end, and process it in real-time to have an uninterrupted view of the performance and availability  of its applications.

    Modernize the Process Landscape

    A perfectly executed application modernization program can fail to deliver results if the surrounding requirements—for example, planning and meeting quality standards—are not met. Legacy processes can be a drag on modernization efforts. It is therefore necessary to address this issue by rearchitecting processes to fit a more automated setup by supporting code developers with teammates who understand the business requirements, adopt best practices and follow the example of successful digital natives. 

    Track Performance Metrics

    Going back to the point about visibility; enterprises need to establish key metrics to track so they can monitor the performance of applications right down to the code level. And it’s not just applications—the entire infrastructure, from cloud to databases to networks, needs to be tracked continuously to identify problems before they blow up. The performance of applications needs to be compared with the corresponding pre-modernization values to understand the benefits as well as identify areas needing optimization.

    Balance and Act

    Application modernization approaches range from rehosting and refactoring to rearchitecting and replacing, which are progressively costlier and riskier, but also progressively more impactful. Enterprises need to decide what’s best for them based on considerations of effort, cost, risk and result. While some enterprises may go slower than others, eventually they will have to modernize all their applications to be competitive and meet their customers’ needs in the most effective way.

  • Zero-Trust Network Access Platforms Slash DevOps Bottlenecks

    Zero-Trust Network Access Platforms Slash DevOps Bottlenecks

    In a prior article, I indicated why a new remote zero-trust platform is needed to support DevOps teams.

    Traditional remote access configurations are not well-suited for DevOps given a shifting user population comprised of employees, contractors and other third parties often using their own devices; the large number of tools and systems to which access must be provided and the hybrid nature of the infrastructure itself (on-premises, hybrid and multi-cloud).

    DevOps Pain Points

    Three pain points are associated with traditional approaches when used with DevOps.

    1. DevOps user experience is poor in traditional remote access configurations. Each DevOps service must be accessed as separate secured sessions, which is confusing and tedious.
    2. System management requires IP whitelisting rules in VPN, static SSH keys in Bastion, firewall rules controlled by different teams and application-specific authentication. The coordination and execution of so many touchpoints for common actions—including the onboarding of new team members, changing roles or adding a new service—invites mistakes that cause productivity issues and introduces unforeseen security holes.
    3. Security is complex and error-prone. DevOps users must be granted restricted access to the individual services they need to be productive. Administering restricted access for individuals and groups is time-consuming. Security policies typically require monitoring detailed audit logs of user access.

    How a Zero-Trust Remote Access Platform Works with DevOps

    In my whitepaper, Zero-Trust-Remote Access-For-DevOps, I explain how a zero-trust remote access platform can resolve the requirements for improved DevOps user experience, consistent and manageable administration and simplified, less error-prone security.

    Improved DevOps User Experience

    By replacing the need for the user to tediously set up separate sessions for each DevOps service they need to use, a client-side DevOps service catalog enables all DevOps services to be accessed with a single click. This service catalog approach is more convenient for DevOps users and provides a form of DevOps governance. It does this by encouraging DevOps teams to select choices that are available in the catalog instead of creating new choices that result in expensive DevOps tool sprawl.

    Simplified DevOps Management

    Management of remote DevOps users is simplified by consolidating access for all DevOps users and services under a single unified service manager and connector. The need to manage each user and service with different teams, each with their own processes, is unified and streamlined when the management requirements for DevOps users and services are consolidated under a common zero-trust remote access platform.

    In addition, this approach makes it much easier to implement more adaptive policies across the breadth of users and services.

    With a zero-trust remote access platform, onboarding new DevOps users and groups is accomplished within a single session—not separate sessions for each onboarding requirement.

    Simplified and Less Error-Prone Security

    Consolidating access to DevOps services under a common zero-trust remote access platform with unified services and services connectors enables simplified and less error-prone security controls, improved security monitoring and policy management. Audit logs of access are reported consistently for all services. Monitoring and changing access rules can be supported for individuals or groups when adjusting a policy. Trust-based access controls can be implemented consistently. Granular policies ensure access is predicated on appropriate device trust score thresholds being met. Rather than a single check prior to granting access, these production services policies continuously assess the security of the users and devices accessing the services.

    Transforming to a Zero-Trust Remote Access platform for DevOps

    Transformations of all types, and especially those for DevOps, are made more certain by following a
    seven-step transformation process, described in my book Engineering DevOps. This process serves as a framework for implementing the zero-trust remote access platform.

    The first two steps, vision and team alignment, involve getting the leaders, key influencers and implementers of an organization to recognize and establish actionable goals for the implementation of the zero-trust remote access platform.

    The third step, assessment, requires taking inventory of the current state of DevOps users, DevOps services and policies that need to be supported by the new platform.

    The fourth step, road map, requires determining the plan for implementation. Typically, it makes sense to start with one application. Start with a set of DevOps users and DevOps services needed for a single application or pipeline which can serve as a model to drive adoption to other applications as the solution is proved and accepted.

    The fifth step, realize, is the step in which the zero-trust remote access platform is installed and set up for the initial model application or pipeline. Banyan Security explains how an example tool vendor helps to simplify this step.

    Capabilities to look in for in zero-trust remote access tools include:

    • Deploys in minutes, with no integration requirements on any existing DevOps services or infrastructure.
    • No need to set up VPNs, open inbound firewall ports or manage DNS.
    • Use existing DevOps user devices without diminishing security.
    • DevOps service catalog provides users visibility and one-click access to DevOps services.
    • Support for bundled services, favorites, and autorun capabilities.

    In step six, operationalize, the procedures for commissioning and using the zero-trust remote access platform are proven, benefits are measured and playbooks are hardened for the organization.

    In step seven, expansion, the zero-trust remote access platform deployment is expanded to other DevOps applications pipelines, DevOps users and DevOps services across the enterprise.

    What This Means

    Traditional remote access configurations are not well suited for DevOps administrators or users given the dynamic nature of services, team members, and environments. A zero-trust remote access platform addresses the need for an improved DevOps admin and user experience, simplified access management to DevOps services and simpler, less error-prone security practices and monitoring.

    I want to express my appreciation for the time spent with the Banyan Security team, who has pioneered many of the concepts outlined in this blog and contributed valuable insight.

  • MongoDB’s Atlas Turns Five

    MongoDB’s Atlas Turns Five

    In June 2021, Andrew Davidson, VP of cloud products at MongoDB, talked with Alan Shimel about MongoDB and the fifth anniversary of launching Atlas, the company’s database-as-a-service offering. The video is below, followed by a transcript of the conversation.

    Announcer: This is Digital Anarchist.

    Alan Shimel: Hi, everyone. Welcome to another segment here on TechStrong TV. Our guest for this segment is Andrew Davidson. Andrew, welcome to TechStrong TV.

    Andrew Davidson: Thanks, Alan. Good to be here.

    Shimel: It’s great to have you on. Andrew, I purposely didn’t say a lot about – I just said Andrew Davidson. I didn’t give your company, your title, or anything. I’m gonna let you do that, if you don’t mind.

    Davidson: Well, thank you very much. Yeah, so I’m Andrew Davidson. I’m VP of cloud products here at MongoDB. I’ve been here nearly nine years. I’m a Silicon Valley native originally, but I moved to New York City and found this amazing opportunity here. MongoDB, for those who don’t know, is actually headquartered in New York City. And it’s been an amazing journey since that time, just seeing this incredible explosive growth out there over this time.

    Shimel: Absolutely. Y’know, if you couldn’t tell from my funny French accent, I’m from New York originally. So I got into tech around 1995, 1996. And at the time, I think the mayor of New York was Mayor David Dinkins, and he had started a project that they were calling Silicon Alley. Right?

    Davidson: There you go.

    Shimel: It was downtown. And I actually had an office – first company I sold – on John Street, 55 John Street. But it was shortly after that – eh, shortly; it was probably seven or eight years after that at least, I bet, that Mongo started in that Silicon Alley, New York kind of cauldron of tech stuff.

    Davidson: That’s exactly right.

    Shimel: Yeah. Oh, I remember. I was in New York then. And it was a point of pride, actually, right? I mean, it wasn’t Oracle. We didn’t wanna be Oracle, right? It was the new guys, the new sequel guys, and they made a name for themselves and did well, right? And so I’m glad to hear you made that move from Silicon Valley –

    Davidson: Totally.

    Shimel: – to New York, and it’s a pretty cool thing. Of course that, as I said, was 15, 17 years ago, right? We’re here to celebrate a birthday/anniversary today, too, and that’s the – If I’m wrong, Andrew, correct me. It’s the fifth anniversary of Atlas from MongoDB, right?

    Davidson: That’s exactly right. We launched MongoDB Atlas, which is our global cloud database service, five years ago this month, and we’re just so excited. We’ve come so far. We’ve reached the psychological milestone this same month of 51 percent of our company revenue growing more than 70 percent year over year. We all know that this is the future of the platform. This is the foundation on which we can deliver really the world’s preeminent application data platform. And the value –

    Compared to where we were, y’know, when you mentioned when MongoDB was coming out of Silicon Alley so many years ago, it’s just a completely different universe today. At that time when MondoDB was first created, I think it was largely about timing. There was this combination of factors, giving developers this document model that was totally native for them, and also democratizing a distributed system right at the time in which cloud was democratizing commodity hardware.

    That combination led MongoDB to explode. But this idea of self-managing databases that we realized six, seven years ago, that was gonna go away. People are gonna move up a whole level of abstraction. It’s gonna be all declarative. An end-point in the cloud you can read and write from, and build wonderful applications – that’s what MongoDB Atlas was all about. And five years in, over 25,000 customers, over 100,000 developers signing up every month, it’s just unbelievable.

    Shimel: It really is. Y’know, I mean, look. Again, having been there, the original attraction with MongoDB was “no SQL,” right? It was a no sequel database. Everybody was using SQL scripts, and whether you were using MySQL or MS SQL, it was – And then the idea of no SQL was really huge. But why was it huge? I think you hit it, Andrew. Number one, it was easier for developers to do what they wanted to do, how they wanted to do it. Number two, it was just lighter and easier and more fungible, if you will, especially as we were moving to cloud in 2005 and 2006, right?

    Cloud burst on the scene. And it lent itself. It lent itself to multi-nodes and all of these things much better than relational SQL database.

    Davidson: Totally.

    Shimel: But of course Atlas comes in as a service.

    Davidson: Absolutely.

    Shimel: And I think that – it’s more than just a service. A database is a service, or data is a service. But fundamentally I think that is the next leap forward, right? We went from like 2001: An Odyssey, right? Someone _____ to throw a rock, and that was a big leap forward. But it was the next leap forward with fire, that defined it.

    Davidson: That’s right. The obelisk, I suppose you could say.

    Shimel: Right, the obelisk, no, totally.

    Davidson: ‘Cause I mean, like you were saying a second ago, why is it so important for developers to have a data model that’s so native for them? Well, it’s because every company today has that incredible sense of urgency that they must be come software companies. In every vertical, every part of the world, everyone’s either fighting to keep what they’ve got or being challenged by upstarts. And you can’t buy this off the shelf any longer.

    You have to bring that DNA in house and really reflect your company’s unique value propositions and users and internal logistics and all of the above for software, and data is the heart and soul of software. It’s always been the hardest part of, frankly, building software. We change how we build software. We layer in microservice as next-gen frameworks, et cetera. But if the data model hasn’t kept up, and if the data model continues to be this sort of tabular model that goes back to the ’70s, a time in which storage was the key bottleneck in computing because it was expensive to have storage then –

    We flash forward to today. The bottleneck is actually our technical people’s ability to understand and build that software. And so MongoDB flips this whole thing on its head and allows those people to be kind of these artisans of the business today, unlocking their ability to really make those businesses competitive. And that’s why I think the strategic value of this is rising. And then when you deliver it in a service, you get so far away from this kind of plumbing of dealing with infrastructure management, operating systems –

    Who wants to be talking about that stuff? It’s non-differentiating, right? You have to move to your custom software, and you have to be able to utilize the best-in-class capabilities. Best in class database as a service globally on the Big Three public cloud, like MongoDB Atlas. Use that with best-in-class other capabilities, and really weave together your unique business, your unique company. That’s what’s happening out there.

    Shimel: Agreed. You said something earlier, too, that wasn’t lost on me, and I wanna make sure it wasn’t lost on the audience. Here on the first anniversary of MongoDB Atlas, it now accounts for more than 50 percent of revenue for the company, which means it’s the dominant revenue-maker for the company. And how long are you working on the Atlas project now, Andrew?

    Davidson: Since the beginning. I mean, really five, six years on Atlas, but at MongoDB almost nine.

    Shimel: Yep. So I guess the question is, and be honest –

    Davidson: Yeah.

    Shimel: – don’t fudge it – did you imagine five years ago that this would be the dominant revenue source for Mongo? Or did you always think it was?

    Davidson: That’s a great question. I think part of me thought, “Y’know, it’s gonna be so hard to get to the point where you can be trusted to power the back end of these mission-critical applications all over the world. This is gonna be such a massive mountain to climb!” I think I had doubts at times, but on the flip side I knew: this is the future of databases. The only way we’re really gonna get to that next level, the only way we truly can disrupt this massive database market, is if we nail it with this service.

    So we knew, and I think we had pretty solid buy-in at every level of the company, that we had to make Atlas successful. In the early days, y’know, as is always difficult, we were kind of a start-up in the company, a whole new go-to-market model, largely self-service led, credit card customers coming in, long tail. We weren’t ready yet to go out to the enterprises on year one. And kind of earning that right, getting battle tested, getting all the security controls, the regulatory capabilities, the sort of the reference wins.

    Y’know, those early customers like 7-11, and Liberty Mutual, and NBCUniversal. Doing great big things on the platform, really showing that you could do it at scale, you could do it safely and securely. And just validating the value prop that we always knew was gonna be there – it just got easier and easier. The flywheel starts moving, and you started feeling like, “Y’know, we really have a shot, here.” And we started doing more and more deals – SMB, corporate, kind of mid-market segment.

    And then probably year three, we started really going big in the enterprise, and you just kinda knew, “It’s gonna happen. We’re gonna do this.” The growth rate was so high. And now, boom. Yeah, you get to that point. The 50 percent chasm is crossed, growing twice as fast as the traditional software, and it’s like, “What a feeling!” I talked to so many customers, even, who are transitioning from kind of being open-source software companies to SAS companies.

    It is a very hard DNA shift to go through. You go from really just writing software that other people run on their own, to being responsible for the business of your customer. And it’s a whole different trust dynamic that’s really important.

    Shimel: Absolutely. Look, congratulations to you and to the whole team with that. It’s an accomplishment, for sure, right? ‘Cause it’s easy to go from zero to one; it’s easy to go from one to two. Two to four is harder. Four to eight is harder. Eight to 16 is yet harder. When you’re talking about 50 percent of the revenue of a company like Mongo, it’s not a zero to one, right? Five years ago, Mongo was already – I guess it was public. Was it public five – ? I don’t know. How long – ?

    Davidson: I think we went public more – no, I think it was more like three and a half years ago, if I’m not mistaken.

    Shimel: It was still a sizeable amount of revenue, right? It wasn’t just _____, y’know –

    [Crosstalk]

    Davidson: Oh, yeah, absolutely. Yeah, yeah, totally. Yeah, yeah.

    Shimel: So that is a mountain to climb. All right. I’m gonna ask you – We’re running low on time. I got a few other things. I’m gonna ask you to put on your visionary goggles now.

    Davidson: Sure.

    Shimel: Here we are, Andrew, talking to Alan five years from now. I hope not. I hope I’m somewhere in Maui or something. But five years from now we’re talking, celebrating the tenth year of Atlas. It’s now 80 percent of Mongo’s revenue, unless something new came up to disrupt it – who knows?

    Davidson: Totally.

    Shimel: But what does Atlas look like five years from now?

    Davidson: Yeah, look. I mean, the whole philosophy of our platform in many ways is it’s fundamentally about user experience. It’s about making it possible for these application developers and operators and enterprises and start-ups, everyone in between – make it easier for them to build modern applications on a modern application data platform. And what we think makes us really unique is, it all starts with this document data model. The document data model is very much the superset data model.

    You can express key value inside a document. That’s just, y’know, a document with a single index. You can express relational inside a document; it’s essentially a flat document. Y’know, spatial, graph – it’s all expressible in this unified interface. So you can do all these data models and all these use cases, which makes it very widely applicable, very multimodal. But then the important step beyond that is to drive multiple workloads behind the scenes.

    The workloads are kind of how you store the data behind the scenes, the algorithms you use to analyze it. Our philosophy is, behind that same unified, elegant interface, you should be able to express all of the wide array of workloads that power modern applications – operational, transactional, the most mission-critical applications; search, which is about human-powered experiences; mobile experiences on iOS and Android devices, synchronizing data back up and down from the cloud to the mobile device –

    real-time analytics; data-like use cases, with object storage economics. All of that should be accessible at your fingertips through this superset data model. And when we think about kind of wrapping all of that up, continuing to deliver it at scale, secure, highly available, highly scalable, multi-cloud service on the Big Three public cloud, we’re just gonna keep taking this vision to the next level. Be laser focused on the user and on delivering what they need to build the next-generation applications of tomorrow.

    Shimel: I love it, man. Good stuff. Andrew, we’re about out of time. For people who wanna get more information on MongoDB Atlas, where do they go?

    Davidson: Well, you can absolutely go to mongodb.com, but you should also check out our upcoming developer conference, mongodb.live, on July 13. Look forward to seeing you there.

    Shimel: Is that virtual only?

    Davidson: Virtual. Absolutely. Virtual conference this year. Next year I’m hopeful, very hopeful, we’ll be back with the real deal, what we used to call MongoDB World, in New York City.

    Shimel: Absolutely. I’ve been there, been there more than a few times.

    Davidson: There you go.

    Shimel: I’m looking forward to it.

    Davidson: Absolutely.

    Shimel: Y’know, I’m gonna be in New York that week of July 13th. I wish you guys were doing it in person. But –

    Davidson: Well –

    Shimel: Next year. We’ll wait.

    Davidson: Yeah, yeah. Look forward to seeing you _____.

    [Crosstalk]

    Shimel: Andrew, congrat – Absolutely. Andrew, congratulations on your fifth anniversary with Atlas, and great work. It’s something to be proud of, man.

    Davidson: Thank you, Alan.

    Shimel: It’s good stuff. Big stuff.

    Davidson: My final comment is, this community’s growing like crazy. Like I said, 100,000 people sign up for Atlas every month. You should sign up today if you haven’t experienced a free forever sandbox cluster, build your Hello World application. We’ve got courses for you to learn from, documentation up the wazoo, because this is big tent community. Thank you, Alan.

    Shimel: Absolutely. Talk to you soon, Andrew. A pleasure. Andrew Davidson, MongoDB. Fifth year anniversary of Atlas – check it out. We’re gonna take a break here on TechStrong TV, and we’re gonna be right back with another guest.

    [End of Audio]

  • DevOps Dozen² Awards 2021 Nominations Now Open

    DevOps Dozen² Awards 2021 Nominations Now Open

    The DevOps Dozen² Awards are back and open for nominations. This past year has been quite a journey and we are very proud of the DevOps community in their ability to pivot, innovate, adapt and think ahead in these challenging times. Some outstanding leaders and vendors have worked nonstop to reach their goals and achieve the highest level of success. We are thrilled to celebrate their hard work and achievements.

    The DevOps Dozen² Awards honor the most dedicated and inspiring thought leaders and innovators who have made significant contributions to the community. For the second year, we have expanded our awards program to include two different DevOps Dozen sections: DevOps Dozen Tools and Services Awards and the DevOps Dozen Community Awards.

    There are 12 awards for each section:

    • DevOps Dozen Tools and Services Awards — Recognizing the companies that develop and deliver outstanding solutions to empower developers, DevOps and IT operations teams.
    • DevOps Dozen Community Awards — Honoring the leaders who have made a positive impact in the DevOps community.

    Nominations are made by the audience. Anyone can nominate a tool or service, although there is a $250 entry fee for the Tools and Services Awards nominations. We implemented this fee last year to cut down on spam and superfluous nominations. However, there is no fee for Community Awards nominations. Please note that commercial tools and services are not eligible for the Community Awards and will not make it through to the finals.

    Finalists will be selected and winners will be chosen from among the finalists. The finalists will be determined by general public voting (weighted for 40% of the final total) and our judges’ selections (weighted for 60% of the final total).

    The last day to submit your nominations is October 22. To vote, follow the steps outlined on the DevOps Dozen² website.

    Who Are the DevOps Dozen² Judges?

    The following DevOps experts make up the DevOps Dozen² Awards 2021 panel of judges, and will review the submissions made by the public and choose the winners:

    • Alan Shimel, founder, CEO and editor-in-chief at MediaOps
    • Jayne Groll, co-founder and CEO at the DevOps Institute
    • Mitch Ashley, founder and CEO at Accelerated Strategies Group
    • John Willis, senior director of the Global Transformation Office at Red Hat
    • Helen Beal, chief ambassador at DevOps Institute and ambassador at Continuous Delivery Foundation

    The 2021 DevOps Dozen² Categories

    DevOps Dozen Tools and Services Awards

    • Best End-to-End DevOps Tool/Service

    Consolidation in the DevOps space has led to a greater selection of platforms offering end-to-end solutions. This award honors the platform with the most diverse and robust solution for DevOps teams.

    • Best DevOps Repo/GitOps Tool/Service

    Git has led to GitOps and it has become an integral part of the DevOps toolbox. This award is presented to the most outstanding Repo/Git/GitOps tool.

    • Best CI/CD Tool

    This award recognizes the best CI/CD tool that helps teams to deliver quality code better.

    • Best Value Stream Management Tool

    Value stream management has come into its own as an important tool for measuring the business value of DevOps and application development. This award recognizes the value stream management tool that most effectively helps visualize the value stream, track metrics and measure improvement.

    • Best Observability Solution

    This award recognizes the best observability solution for observing and monitoring to understand the performance of individual components and systems and entire applications.

    • Best DevSecOps Solution

    This award honors the best DevSecOps solution, which enables security to be included earlier and continuously throughout the SDLC.

    • Best Testing Service/Tool

    Testing is a mainstay in the DevOps toolchain. This award recognizes the best testing tool for ensuring the quality of code.

    • Best Kubernetes Platform/Service

    The adoption of containers has paved the way for an entire solutions ecosystem supporting and enhancing the container technology. Kubernetes is perhaps the most widely recognized and utilized tool in the space. There are many platforms and services that offer Kubernetes-based solutions. This award honors the best Kubernetes platform/service available.

    • Best Cloud-Native Security Solution/Service

    The cloud-native movement has taken the IT world by storm due to the depth and breadth of the solutions available and the benefits of building applications and services specifically for cloud-native environments. This award recognizes the best tools, solutions and services considered cloud-native.

    • Best DevOps for Mainframe Solution
      Mainframes are far from being the dinosaur of the IT world, and a variety of solutions are available for enabling DevOps practices in mainframe environments and hybrid environments. This award recognizes the best of them.
    • Best DevOps for DataOps/Database Solution
      Databases often have been left out of the DevOps conversation, but that is changing. The whole concept of DataOps is gaining greater traction. This award recognizes the best solution in the database for DevOps and DataOps categories.
    • Best New DevOps Tool/Service Provider
      This award recognizes the best new service provider or tool that has brought innovation to the DevOps space in the past year (must be less than 24 months old).

    DevOps Dozen Community Awards

    • Best DevOps Industry Implementation
      This award honors organizations that have successfully adopted and implemented DevOps principles and practices.
    • Best DevOps Survey/Analysis/Research
      This award recognizes research that has significantly and positively impacted the DevOps community.
    • Best DevOps-Related Video Series (video)
      This category highlights the most entertaining, educational and inspirational DevOps video series—those with the most engaging and highest-quality content.
    • Best DevOps-Related Audio Podcast Series
      Which is your favorite DevOps podcast from this year? This category celebrates the many great DevOps podcasts sparking some interesting discussions on a multitude of topics from CI/CD and testing to DevSecOps, Kubernetes, containers and more.
    • Best DevOps Book/eBook of the Year
      Each year sees a plethora of well-crafted DevOps-related books and ebooks. This category highlights the most popular and best read of the year.
    • Top DevOps Evangelist
      This category is one of our favorites. There are so many great evangelists who spread the DevOps gospel and we want to know who does it best. Nominees can work within the DevOps space or outside of it.
    • Best DevOps Transformation (non-vendor)
      In a year in which digital transformation became a must-have for survival, we want to know which organizations stood out in their efforts to transform their business through DevOps.
    • Best DevOps Virtual Event of the Year
      COVID-19 has changed the game when it comes to hosting events. In 2021, virtual events remain the norm, reaching a global audience while everyone stays safe at home. Which DevOps virtual event stands out from the others?
    • Most Innovative DevOps Open Source Project
      Open source projects are meant to promote innovation and increase efficiency, velocity and quality. While there are many great open source projects out there, the best ones are those that create a community of people who work on and with the technology and support each other.
    • DevOps Executive of the Year (non-vendor) This category celebrates excellence among DevOps leaders at the C-suite (or equivalent) levels.
    • Best DevOps Presentation of the Year
      This category recognizes the best DevOps-themed presentation from the many (virtual) conferences during the year.
    • Best DevOps.com Article of the Year
      DevOps.com publishes multiple new articles every day. Which was the best article from 2021? Let us know so we can recognize the author.

    All the solutions nominated to the Tools and Services Awards will automatically be registered into the DevOps Tools Guide authored by MediaOps and Accelerated Strategies Group. This initiative is our effort to create a comprehensive listing of tools and tools vendors to help end users better understand the DevOps market landscape.

    Choosing a winner is difficult when there are so many talented and inspiring leaders doing an incredible job to empower and support our community. Although not everyone can win an award, being nominated says a lot about each and every one of you. We greatly appreciate your dedication and service to the DevOps community.

    For more information, please visit the DevOps Dozen² website. If you have any questions regarding entry process, awards, timing or any technical issues regarding the website, please contact us at DD@devops.com.

    Alan Shimel

    CEO, MediaOps

  • Choosing the Right Metrics for DevOps Adoption

    Choosing the Right Metrics for DevOps Adoption

    DevOps adoption has skyrocketed in recent years. Its impact on traditional development platforms has long been well-documented, but in more recent years it has started to influence low-code platforms, too. In its recent survey on the state of Salesforce DevOps, Gearset found that 69% of teams are already using source control and another 20% plan to start this year. Additionally, 45% of companies have implemented CI/CD, and that number is set to increase by 33% this year. However, for high-performing developer teams to recognize the benefits of DevOps and strive for further DevOps adoption, they must use the right KPIs.

    While it can be tempting to take a ‘big bang’ approach to DevOps adoption, teams see more success when they take it one step at a time, with each stage building on the last.

    That said, to experience the full ROI of DevOps adoption, it’s still critical to see the journey through to the end. It’s not uncommon for teams who begin the process of DevOps adoption to stop halfway through or, worse still, lose motivation and halt the process entirely. DevOps adoption is a process full of technical and cultural change, and one of the most common causes of its failure is an emphasis on the technical at the expense of the cultural.

    The best way to tackle this is to encourage teams to take a product mindset and apply it to DevOps adoption. Simply put, rather than tasking your teams with “adopting CI” for instance, they should be asked to solve a specific business problem. For example, “When we finish work on a story, it takes weeks for it to get released!” It sounds obvious, but focusing teams around business impact rather than on individual tasks makes sure that there’s a direct correlation between the work that’s happening and the desired business outcome. This is the crux of adopting a DevOps mindset, rather than a purely technical and perfunctory approach to DevOps adoption.

    One of the key things a team can do to drive this type of change—and to avoid common causes of failure—is to use the right metrics at each stage of their DevOps journey.

    There are a number of well-recognized metrics for measuring DevOps success, including:

    • Deployment time
    • Change failure rate
    • Release cadence
    • Lead time
    • Recovery time

    These metrics serve two purposes. They’re well-documented and benchmarked indicators of performance that can help teams focus on their biggest problem areas. And they’re measurable outcomes that prove the business benefits of DevOps, such as agility, improved collaboration, higher quality systems and resilience, not to mention significant time and cost savings and efficiency.

    Choosing the Right Metrics

    Teams must choose the right metrics at the right time to identify their opportunities for improvement and to measure the impact of each stage of DevOps adoption. The right metric empowers a team to solve their most pressing challenges, and to demonstrate the value of their work to their leadership.

    Here’s how the benefits of each stage of DevOps maturity should be measured: 

    Stage of DevOps maturity

    Most relevant metric(s)

    Choosing your release tools

    Deployment time

    Adopting version control

    Change failure rate

    Recovery time

    Automating testing

    Change failure rate

    Automating deployments (CI/CD)

    Release cadence

    Lead time

    Integrating backup into DevOps tools and process

    Recovery time

    Let’s expand on each stage and the appropriate KPI.

    Choosing Your Release Tools

    This is more relevant to low-code platforms, but still broadly applies to the wider tech world. Even in the most mature of development processes, there will be manual steps. Getting changes into version control, reviewing changes, resolving conflicts, triggering releases—each of these is a manual step, and the effort involved in each of these manual steps is heavily dictated by the tools you use.

    Finding the right tool for each stage of your development and release process can therefore have a massive impact. The right deployment tool, for instance, can cut down deployment time by as much as 90%. This step alone increases efficiency and ultimately saves companies vast amounts of time and money. In extreme examples, teams that save so much time by picking up a couple of new tools to patch up their manual processes can be tempted to stop there. This is why teams need to look at other metrics as they proceed to adopt more DevOps processes.

    Adopting Version Control

    For most teams, a world without version control is unthinkable, having built their development and release processes around it a decade or more ago. For others, particularly in the low-code space where it’s yet to reach ubiquity, it can feel like an unnecessary burden. The introduction of version control is an extra step in the releases process and another level of complexity, with new tools and practices to learn. But version control is the foundation of an automated DevOps workflow. This step allows for more team collaboration and visibility into that collaboration, with code reviews and a detailed change history. Version control also allows teams to deal with merge conflicts early, rather than in the middle of a release. 

    After adopting version control, teams will see a reduction in their change failure rate due to code reviews and their recovery times will be shorter as version control offers rollback and a quasi-backup of the code and config captured by their version control system. But the real benefits of version control are in the DevOps processes it unlocks. 

    Automating Testing  

    It goes without saying that the purpose of testing is to improve the quality of releases. It does this in a few ways:

    • Unit testing – Exercising individual methods and functions in isolation generally results in more maintainable code, which has positive benefits for developer productivity and even things like handling merge conflicts.
    • Static code analysis – While this is not testing in the strictest sense, automatically verifying that code adheres to a set of best practices similarly improves code maintainability.
    • Integration testing – This type of testing ensures that combinations of classes interact and generate the expected results when combined together. This brings another level of validation that changes being made are likely correct.
    • End-to-end testing – UI testing is an example of this; exercising the system in the way a consumer would and verifying that the results are deterministic and correct. This also includes regression testing for key workflows and sometimes even acceptance testing.

    The more of these testing approaches a team uses, the more confidence they’ll have in their changes and the less human verification and intervention will be required. As teams’ DevOps processes mature, it’s often the human element that becomes the biggest bottleneck, so confidence in deploying changes is very valuable.

    When teams introduce more testing and test automation to their release pipeline, the right metric to watch is change failure rate—the proportion of releases that include a bug or error. The immediate benefit of higher-quality code translates to savings in the long term as less time is spent finding and fixing bugs or reworking code.

    Automating Deployments (CI/CD)

    If choosing the right deployment tool significantly reduces deployment times, automating deployments with a CI/CD workflow eliminates the time it would usually take to run those deployments manually. However, deployment times aren’t the key metric for measuring the success of CI/CD implementation. The most important metrics to pay attention to here are release cadence and lead times, which are dramatically improved thanks to painless, automated deployments. 

    With automated deployments, teams can increase their release cadence; rather than releasing huge batches of changes once a month, they might begin to release smaller packages every week. Ultimately, the goal is to have a steady stream of incremental improvements being released on-demand, and usually multiple times per day. Accelerated release cadences then result in reduced lead times—the period between work being finished and finally released to production. This is beneficial for developers and their end users as more frequent releases create more opportunities to act on user feedback earlier in the process. 

    Integrating Backup Into DevOps Tools and Process

    Recovery times are a key metric for DevOps success, as robust DevOps processes will improve the speed at which teams can restore after an incident. For even better resilience and faster recovery times, teams should integrate backup into their DevOps tools and processes.

    Restoring from a backup is comparable to a deployment, and this means that restore processes also can harness the power of DevOps so they feel like a familiar process—not something unusual that teams struggle to execute at the point when it matters most. It’s also important to make it as easy as possible to back up on demand just before a risky release.

    Downtime and data loss are massive costs to any business, so improving recovery times is a huge benefit of a fully mature DevOps process.

    See the Benefits of Maturing your DevOps Process

    When teams pay attention to the right metrics, they understand the benefit of each DevOps process individually and they are motivated to keep maturing their DevOps processes as a whole. It’s all well and good to take the pain out of deployments and releases, but high-performing DevOps teams effectively make deployments an asset for releasing more often and getting feedback earlier. As a result, they are better equipped to deliver more value to their companies—sooner.

  • 4 Benefits of Integrating a Language With a Database

    4 Benefits of Integrating a Language With a Database

    One of the hallmarks of a truly modern programming language is the coupling of a language—for creating applications, devising algorithms and solving business problems—with an underlying database. When there is practically no separation between a database and its programming language, developers immensely increase their productivity, maximize the effectiveness of the code they write and provide unparalleled speed and flexibility in supporting business needs.

    Predicating a programming language on a database delivers these advantages by enabling developers to treat data as if it is in-memory, liberating them from the burden of manipulating how data is represented in the underlying database. This one simple advantage exponentially increases the flexibility for addressing business problems and, when used with advanced logic languages, enables developers to write applications in a fraction of the time and with less effort than it would otherwise require.

    Specific benefits of tightly coupling a database with its programming language include eliminating the time-honored mismatch between data structures and database representations, smart caching with transactional technologies for committing or rolling back database changes and automatically changing object definitions.

    These benefits remove the database as an otherwise complicating factor in supporting business use of data by freeing developers to flexibly innovate solutions—and speedily implement them.

    In-Memory Manipulations

    Developers focus on manipulating data structures to solve business problems. Traditionally, this concern was hamstrung by the onus of translating those structures into how data are represented in the database, a time-consuming necessity detracting from programmers’ attention to business problems. By fully integrating an object-oriented database or graph database with its programming environment, users can manipulate data objects in memory and they’ll automatically persist to storage. Developers can solely focus on working with data structures for business objectives without spending just as much, if not more, effort wrangling data’s representation in the database.

    Programming languages create this effect when they are imbued with an intelligent caching layer that persists in-memory data manipulations into the database. This advantage requires basing the programming language on a specific database. It eliminates the mismatch between working on data structures and toiling to get the underlying data to and from disk.

    Smart Caching

    The caching layer also tames the inherent difficulties in data objects pointing to other objects, which is a hallmark of object-oriented systems and graph databases. These databases require intelligent caching because one object can conceivably include the majority of the database by pointing to objects that, in turn, point to additional objects. Tight coupling between that language and its database fixes this issue so developers cache what is needed to work on data in memory while automating its impact on disk. Such caching is significantly improved with transactional technologies so developers can specify when they want their modifications saved in the database.

    This way, if they make a mistake they can simply go back in time before it was made and undo it. Conversely, after successfully completing their work, they can easily commit changes to the database. Like the other advantages, this benefit boosts developer productivity to match that of business requirements. It adds to the flexibility programmers need to innovate solutions for mission-critical objectives by writing or integrating applications.

    Object and Class Definitions

    The third boon of basing a smart programming language on a database is the capacity to dynamically change the definition of data objects. With conventional approaches, this task requires developers to stop writing or operating applications and manually write code to adjust the classification, which may involve either reading or writing the data in and out again. But with no separation between the database and its sufficiently smart programming language, such coupling can effectively keep track of classifications while automatically modifying them as needed.

    Developers can seamlessly add or delete features without manual coding, which also furthers their productivity. For example, if they add an attribute to an object, the next time they see it it’ll magically have the new feature. Alternatively, if they create a new feature of an old classification and decide to delete it, they won’t see it anymore. Best of all, these changes occur without rewriting the classification.

    Advanced Logic

    There is one more capability of very smart programming languages: They can combine these advantages with advanced logic languages such as Prolog. Prolog, or a subset of Prolog called Datalog, can instantly be used as a query language. In addition, it becomes very easy to create rule-based applications with Prolog because one can write declarative, logic-based rules with minimal coding. For example, they can specify a series of event-based rules detecting customer churn, money laundering or any important business outcome without being impeded by unnecessary database concerns to deliver time-to-value that’s unmatched without this method.

    Basing a programming language on a database improves developer efficiency and effectiveness by letting developers work on data in memory and persisting changes to storage, intelligently caching data and rolling back changes as required, and dynamically adjusting definitions.

  • Avoiding Budget Overruns During Cloud Migration

    Avoiding Budget Overruns During Cloud Migration

    Many organizations have already started their migrations to the cloud, or soon plan to. It makes sense: The cloud is an attractive option for multiple reasons, including its potential for cost savings, scalability and support of continuous application development.

    However, benefits aside, it can be very easy for enterprises to lose financial control of a cloud migration. The potential for costly mistakes when shifting to the cloud is significant, and organizations often end up getting hit with surprising bills that ultimately stop them in their tracks. In fact, in a 2020 survey, 94% of participants claim that it is a challenge for them to take full advantage of public cloud resources, with 40% citing the need to control cloud costs as their biggest obstacle.

    These financial mistakes are typically made due to the pay-per-use model of cloud services and the growing cloud skills gap. Too often, IT team members are forced to work in cloud environments they are not proficient in, and as new features are released every week, it’s almost impossible for these individuals to keep up. One recent study claimed that 90% of enterprises found that their cloud skills gap doubled over a three-year period.

    The Two Different Types of Cloud Migrations

    There are two types of cloud migrations that typically occur, a lift and shift, or a replatform/rearchitecture migration. The lift and shift method is fast and relatively easy, but it doesn’t enable organizations to truly optimize their cloud usage. The rearchitecture method is the most popular as it gives an organization the opportunity to fix any outstanding issues by harnessing the benefits of cloud-native technologies and services.

    However, once a rearchitecting migration begins, organizations often find it takes significantly more time, effort and resources to complete than they originally thought. This can become a real challenge, especially when facing critical deadlines or experiencing major events like the COVID-19 pandemic or the wild weather events experienced around the globe.

    Understanding and Assessing Performance

    For both types of cloud migrations, understanding and assessing performance and underlying architecture is an ongoing task, as cloud providers are always releasing new or improved services or lowering prices. To make sure they’re achieving the best performance and building a resilient architecture, application developers and IT teams must always be looking for ways to improve their applications. Within the cloud, this is typically focused on four key areas: Networking, computing, storage and application architecture.

    From a network perspective, understanding where data is being used and where data is being accessed is critical and can help organizations cut costs exponentially. For example, cloud providers often have costly data ingress and egress charges, so assessing how best to access and use data when necessary can help a company navigate those charges.

    Regarding computing, there are multiple resources that can help manage system load, make tweaks to computing infrastructure and find performance and cost benefits. For instance, leveraging the data and metrics from native cloud reporting tools can help organizations understand and predict demand, and taking advantage of services like auto-scaling enables them to scale up or down appropriately depending on the changes in application use and demand.

    On the storage front, cloud providers offer a number of ways to save money and boost performance. From the release of GP3 volumes to tiered long-term storage options based on the frequency of access, storage is an easy way to find cloud savings opportunities. Additionally, cloud providers will usually make recommendations and set up tiering policies to help move data automatically into colder and deeper storage.

    As cloud providers continue adding new services, looking at your application architecture is essential to innovation and cost savings—especially when including labor and effort—and truly achieving the promise of the cloud.

    Solving Common Costly Mistakes

    While there are numerous examples of common financial mistakes companies make when managing cloud instances, lack of visibility and understanding are the leading pain points for the developers who stand up and maintain their organization’s architecture. With the demand for cloud so high and with their attention directed elsewhere, developers and engineers often take the mentality of “If it isn’t broke, don’t fix it.” This hampers the company’s ability to assess and optimize their cloud operations. They don’t have time to consistently review their applications for ways to save money or even know how much their decisions cost.

    This ubiquitous issue can be solved by bringing greater visibility into the cloud migration and management process and aligning real-time finances directly with established corporate budgets. In addition, organizations struggling with the cloud services’ pay-per-use model should take the necessary steps—whether internally or with a third-party vendor—to ensure cloud resource and service use is being effectively monitored. This makes it easier for developers to understand where and how their actions affect the financial posture of the organization before their boss comes knocking on the door.

    If enterprises provide the resources necessary for IT teams to become more knowledgeable about how their applications are being used within the cloud and address the cloud skills gap in their workforce, they will be able to prevent cost overruns, experience cost savings and take back control of their budgets. This can take a variety of forms, but enterprises need to establish continual training programs to ensure team members are up-to-date on the latest cloud features and services, as well as leverage tools and automation to reduce manual intervention. Providing this critical visibility throughout a cloud migration can often mean the difference between a successful, on-budget migration and one that is stalled and costly.

  • 3 Pillars of Autonomous IT Ops

    3 Pillars of Autonomous IT Ops

    Imagine a future where human IT operators and intelligent systems work together in a virtual operations room; the systems do the heavy lifting while the humans supervise. This may seem like science fiction, but the truth is we are already on the way to that future with AIOps. While actual AI-augmented teams are still quite a way off, AI/ML tools are slowly but surely becoming more ubiquitous within IT Ops, NOC, DevOps and SRE teams and leading us into this not-too-distant future.

    In the last five years, the number of internet users grew by 50%. In the last two years alone, more data was generated than in the entirety of human history up to that point. Today, 5 quintillion (that’s 18 zeros) bytes of data are produced by smart devices daily. These massive volumes of data are having a huge impact on digital operations.

    Looking at the enterprise, we see similar trends: More tools generating more data and with it, greater complexity. This is due to many factors, including the transition to the cloud, M&A activity that introduces new tools and data into an existing organization and the adoption of new technologies and methodologies such as CI/CD, microservices and more. At some point, there is so much data and everything becomes so complex that IT Ops teams reach a breaking point where they can no longer manually manage the data being generated.

    And when that point is reached, outages and service disruption become an everyday reality, as IT teams are pulled away from their strategic, business-critical projects into what is essentially firefighting. 

    Implementing more observability and monitoring tools simply exacerbates the problem, and scaling teams isn’t cost-effective in the long term. The more forward-thinking solution is automation, and even more so, autonomy. 

    The Road to IT Ops Autonomy 

    Autonomy is the ability of IT Ops to operate with minimal to no human intervention. It relies on the automation of the incident management life cycle so that humans are needed only for supervision, in the same way that a single operator can supervise a modern and highly complex robotic car assembly line.

    How can we hope to reach IT Ops autonomy?  The following three proposed pillars can act as a guide: 

    Pillar One: Automate

    Pillar Two: Predict and prevent 

    Pillar Three: Democratize

     Pillar One: Automate

    We must automate as much as possible across every step in every aspect of incident management. It won’t happen overnight, but with time and with the help of AI/ML it is doable.

    Consider the autonomous driving analogy.

    Just a few years ago, cars were manual, with little to no automation. Then we moved through partial autonomy to where we are today–conditional autonomy–where the car can drive itself in ideal conditions. This is a level of automation that we are beginning to accept; some of us cautiously, others with more enthusiasm! Meanwhile, progress continues toward higher levels of autonomy, in which vehicles will be able to drive themselves in most conditions. One day, maybe, we’ll get to the holy grail of full autonomy, where the car will drive on its own–perhaps we won’t even need a steering wheel!

     

    Similarly, in IT Ops, automation is a process and it will take time to build not only the capabilities required but also the necessary trust in AI/ML technology. Such a journey typically begins with human-driven automation, as “if-then-else” types of automation are added manually. For example, if a certain alert appears, a ticket will automatically be assigned to a specific person. As our confidence grows, the next stage could be AI-driven suggestions, whereby AI applies what it has learned from previous human actions to suggest what to automate, but the human expert decides whether to accept the suggestion or not. 

    At some point, we will progress to allowing AI to partially automate some tasks—perhaps those that are less risky, such as collecting diagnostic information and adding it to the alert, so that the incident responder automatically receives both together. We may even automate some parts of our system; perhaps within a staging environment that isn’t production-critical and, as such, is a good place to validate that the automation is working as expected. Over time, as our AI and ML capabilities develop and our trust in them grows, we may eventually reach full AI-driven automation, with no human involvement at all—the AI will simply detect the fault and resolve it. 

    Pillar Two: Predict and Prevent 

    Typically, when assigned an incident, a responder will take note of the incident’s priority. But they will also rely on their common sense and past experience in deciding how to respond. So, for example, if a P3 incident had recently escalated to a P1 in 30 minutes and a similar P3 incident now appears again, our responder will probably prioritize it, even over other incidents, to avoid a similar escalation.

    While this is one of the advantages of human logic, the flip side is that the IT environments in which we work are vast and complex—no human can possibly remember every potentially relevant detail. What’s more, if a different responder dealt with the earlier incident in our example, the current responder would not have the same insight. What is needed is an element of enrichment that can input leading indicators into the platform for the next operator to see and act upon to prevent an incident.

    This is where AI can help. 

    By identifying patterns in the data haystack—linking different incidents to the same root cause, for example—AI not only generates insights that help resolve these incidents faster but can also predict a certain outcome. This is done by anticipating a future chain of events so that appropriate preventative action can be taken before the event occurs. For now, this preventative action may be manually taken by the operator, but at some point in the future automation may enable the problem to be identified and fixed by the AI with no human intervention before it results in an outage.

    Pillar Three: Democratize

    The last pillar that will help us achieve full autonomy in IT Ops is democratization of AI. AIOps should not just be accessible to advanced, mature organizations with an army of data scientists working behind the scenes. Just as anyone can accomplish almost anything online these days with just a few clicks, any organization should be able to adopt AIOps and enjoy its benefits! 

    Democratization is about making relevant information accessible to anyone who needs it, when they need it, and in a manner that is easily applicable and actionable. AI should be accessible through simple user interfaces, with no need for coding; the benefits of machine learning should be available to people that don’t (need to) understand machine learning models. 

    This democratization can be achieved by making AIOps platforms simple and accessible to all—from the admin to the user. For example, an AIOps platform might automatically suggest how admins should prioritize incidents for operators, to make triage easier and provide instructions that even a non-expert can follow. The more such features are built into AIOps platforms, the more democratized AIOps will become.

    Data volumes will continue to grow exponentially in the coming years, and the only way to eventually conquer the tyranny of overwhelming data will be to reach autonomy in IT Ops. 

    If you’re an AIOps vendor, keep the three pillars in mind so your customers can continue to progress towards autonomy.  If you’re an IT Ops leader or tooling architect, and IT Ops autonomy is part of your vision, make sure that your AIOps vendors’ platforms and tools have been architected with these pillars and their principles in mind. 

    How far are we from achieving autonomy? Only time will tell. But one thing is for sure—we will get there, and if the COVID-19 pandemic has taught us anything it’s that we may be seeing machine-augmented teams sooner rather than later. 

  • Google Donates Sqlcommenter to OpenTelemetry Project

    Google Donates Sqlcommenter to OpenTelemetry Project

    Google announced today that Sqlcommenter, an open source project that provides access to an object-relational mapping (ORM) auto-instrumentation library, is being merged with OpenTelemetry, an open source project for building agent software to instrument applications.

    Nimesh Bhagat, a product manager for Google, said Sqlcommenter is designed to make it simpler to correlate application and database telemetry. OpenTelemetry provides a mechanism for collecting logs, metrics and traces that is being advanced under the auspices of the Cloud Native Computing Foundation (CNCF).

    The challenge organizations using OpenTelemetry to collect that data will face is the need for tools to query that data in whatever platform it ultimately resides, noted Bhagat. OpenTelemetry previously lacked a common standard by which application tags and traces can be sent to databases and correlated with an application stack. Google envisions that IT teams will either employ Sqlcommenter to explore data or access via platforms such as Cloud SQL Insights or a third-party application performance management (APM) platform.

    Google made Sqlcommenter available as open source software earlier this year. It is designed to enable object–relational mappings (ORMs) to augment SQL statements before execution by, among other things, providing access to comments containing information about the application code that caused its execution.

    Sqlcommenter also allows OpenTelemetry trace context information to be propagated to the database, enabling correlation between application traces and database query plans that ultimately should make it easier to, for example, determine the root cause of a performance issue involving a microservices-based application making calls to a database. Sqlcommenter is primarily designed to be employed by full-stack developers that have some familiarity with how to use SQL to query data residing in an observability platform, noted Bhagat.

    Specifically, Sqlcommenter is designed to provide deeper insights into the performance of backend databases by simplifying the process of correlating slow queries with source code. It currently supports Python, Java, Node.js and Ruby languages along with ORMs such as Django, Sqlalchemy, Hibernate, Knex, Sequelize and Rails.

    As the number of applications that are instrumented using open source software continues to steadily increase, the amount of data that a DevOps team could potentially query also will steadily increase. Cloud storage platforms have made it easier to store that data at an affordable cost but the next challenge is clearly going to be finding a way to surface meaningful insights from all that data. Undoubtedly, achieving that goal will require everything from query tools to machine learning algorithms capable of identifying patterns at a level of scale that would not otherwise be possible.

    In the meantime, DevOps teams can look forward to being able to attain a level of observability that previously could only be achieved at a cost that tended to limit the number of applications organizations would instrument. At a time when the dependencies between applications increases with each passing day, that level of observability will soon prove indispensable.