Category: Blogs

  • Driving Software Delivery Automation With AI/ML

    Driving Software Delivery Automation With AI/ML

    What if we could use an artificial intelligence (AI) engine to predict the next lines of software code to build? What if machine learning (ML) could trigger the creation of automated test cases to validate our solutions?

    Automation has been a focus for the information and communications technology (ICT) industry for years, and there are some lessons that could apply to the software developer community, too, as well as other verticals. Today, more streamlined execution of operational and software delivery processes, such as continuous integration, continuous delivery and continuous deployment (CI/CD) in app development, continue to drive new levels of cost reductions, allowing products to get to market and update faster. 

    However, with the increased amount of data comes more complex ecosystems and the need for the next level of automation. Hyperautomation, or the use of AI/ML to further streamline processes, could build on these CI/CD processes, revolutionize the way multiple industries deliver software and optimize operational processes.

    Today’s Network Complexities and Challenges

    In the telecommunications industry, disruptions from hyperscalers and other innovative players, as well as increasing levels of investment in 5G technology, are forcing the hand of communications service providers (CSPs). This is causing them to turn to automation to improve their levels of operational efficiency and innovation to keep up with the need to push frequent updates. 

    However, challenges remain in the form of:

    • Total cost of ownership and operating costs: With more frequent needs for upgrading existing networks, additional levels of automation including AI/ML and hyperautomation are required to manage and maintain today’s complex networks. 
    • Time to market: In an increasingly competitive market, the CSP’s ability to react to market needs and/or new marketing ideas and to rapidly launch those needs and ideas to their end users and enterprise customers is a key competitive differentiator. 
    • Customer experience: Digitalizing business processes, providing an increased level of self-service and seeking ways to optimize customers’ overall journey remains an important need.
    • Employee productivity: In an effort to increase revenue, more efficiently drive their networks and improve end user customer experience, CSPs also need to increase employee productivity. They can do this by engaging their employees to solve bigger problems. As a result, the simplest and most repetitive low-level tasks can be automated.

    The Power of AI and ML

    The automation journey started with breaking down complex processes into smaller parts and automating discrete parts using scripts, runbook automation (RBA) and/or robotic process automation (RPA). 

    With the introduction of AI emulating human intelligence through mechanical and computational processes and ML using algorithms that learn from data to make predictions and decisions, CSPs can now bring automation and hyperautomation to the next level. 

    The potential use cases for the delivery and maintenance of telco software solutions are unlimited, and many of these lessons can be applied to other sectors. Some examples include:

    Autonomous Fault and Anomaly Detection for Cloud-Native Solutions

    Today’s network solutions are deployed on containers and microservices. The identification of issues and assurance of service quality becomes increasingly challenging with thousands of workloads running in parallel. Container failure can cause severe service disruptions. In fact, a single service failure or abnormal behavior could have a butterfly effect on the overall system and negatively affect customer satisfaction. 

    A proactive, AI-based monitoring solution can continuously search for anomalous behavior in the environment. Different container metrics and logs can automatically analyze and detect problems proactively and help find the right corrective actions for the application workflows. This is based on identifying anomalies in the container orchestration metrics related to pods, workloads and namespace. These techniques enable a more reliable and predictable application runtime environment with higher service quality and faster identification and resolution of system faults and, ultimately, a better overall customer experience

    Machine Learning-Triggered Closed-Loop Automation

    Large clusters of compute, storage and networking resources are managed throughout a data center to enable the deployment of custom workloads. Due to the inherent nature of applications running on the virtualized and cloud infrastructure, there are usage highs and lows which need constant infrastructure component deployment or reconfiguration of the cluster to ensure cost-efficient service availability. 

    A machine learning-based solution can detect anomalies in the infrastructure. Specifically, with closed-loop automation, enterprises can bring in a layer of intelligence over the current setup to monitor application performance and infrastructure metrics and identify anomalies that require immediate system attention. If those are found, they can trigger the automated addition of capacity to the infrastructure. When anomalies have subsided and once the utilization is capable of being handled by the original capacity, the solution can consistently monitor metrics to roll back the virtual or cloud-native functions. This allows for an efficient and fully automated use of network capacity, minimizing human oversight.

    Smart Code Recommendations

    One of the most clear-cut ways tools can reduce the effort of software development for programmers is through code completion. AI-powered code completion is a convenient way to access descriptions of functions and their parameter lists. These solutions also speed up software development by reducing keyboard input and the necessity of name memorization. They also allow users to refer less frequently to external documentation, as interactive documentation appears dynamically in the form of tooltips. AI-powered code completion can also use an automatically generated in-memory database of classes, variable names and other constructs or references based on machine learning to open code repositories.

    AI-Driven Test Recommendation

    Significant time is spent understanding solution requirements and identifying critical scenarios for testing. This is usually followed by a manual test plan creation and execution. To solve this, AI/ML techniques can be applied to identify what tests will need to be done during each cycle of the software development stage, including automating the analysis of required documentation. This can be used to increase overall test-cycle time and improve productivity. 

    AI-Enabled Test Automation

    API testing is critical for validating application logic and needs to go beyond the GUI layer to validate the core of an application. This implies complexity because the protocols are usually difficult to understand, the API inventory is usually large and sequencing of the API calls is critical to a successful test specification and execution. With AI/ML and natural language processing techniques, the analysis of the solution test plan can be automated to obtain recommendations on what test frameworks to utilize, identify and suggest the parameters for each test case, identify the API call sequence and automate the development of test cases based on reusable functions. This can increase overall quality of service, simplify change management, improve the productivity of the testing team and offer more reliable test cases.

    Test Failure Analytics

    During execution of automated test cases, multiple failures can be observed in various logs. Manually analyzing the logs and correlating them with the application issues can be time-consuming. Also, the identification of unreliable tests impacting system stability is complex. 

    AI/ML techniques such as vectorization, K-means clustering or topic modeling can be applied to automate the collection of test logs across applications and label failures including the type of cluster failed tests. The test subject matter expert (SME) is provided with a failure analysis report summarizing issues and problems automatically from many logs across applications, simplifying the identification of issues in test automation areas or scripts. These techniques can significantly increase test productivity and overall troubleshooting time.

    Accelerated Root Cause Identification and Analysis (RCI&A)

    When a problem occurs in a telco network, it tends to cause a ripple effect, affecting other integrated services and creating an alert storm. In such scenarios, root cause identification can become complex and cumbersome. Event aggregation and synchronization are needed to reduce environment complexity and variance. 

    Machine learning weighted analysis techniques can be used to determine the influence individual IT metrics have in affecting the overall health and performance of hosted applications and services so that pinpointing the root cause of any service impact in real-time can become easier for the team operating the network. Efficient AI/ML techniques for accelerated RCI&A can lead to significant timeline reductions. 

    AI/ML Driving Operational Innovation and Market Agility

    In telco, there is great potential for AI/ML-driven use cases to further improve the cloud-native journey and enable an IT-like software life cycle management process for the solutions of the future.

    More broadly, highly automated and self-healing networks will drive higher levels of efficiency and customer satisfaction. This includes faster testing with less resources; more efficient utilization of network resources and cloud infrastructure; faster time-to-market with more software releases per year with similar capacity as well as an increased quality for faster and more agile monetization of the CSPs capabilities.

  • EP 20: DevOps for Low-Code – Friend or Foe?

    EP 20: DevOps for Low-Code – Friend or Foe?

    Low-code development has become increasingly popular since the onset of the COVID-19 pandemic in 2020. Low-code technologies allow organizations to build and customize fully responsive, easy-to-deploy applications with little to no coding. It accelerates digital transformation and empowers citizens developers. In this episode of DevOps Unbound, hosts Alan Shimel and Mitch Ashley are joined by Derya Sousa (Kianda), Andrew Manby (HCL) and Brian van den Brink (Valori) to discuss the benefits of low code and how it is transforming software development. The video is below and a transcript of the conversation follows.

    Alan Shimel: Hey, everyone, welcome to another episode of DevOps Unbound. DevOps Unbound is a biweekly panel session on all things DevOps, of all things—hence the name, DevOps Unbound. It is sponsored by our good friends at Tricentis, and many thanks to Tricentis for sponsoring DevOps Unbound.

    For those of you who are catching DevOps Unbound for the first time, in addition to our biweekly panel sessions like this, which are pre-recorded, we also do a monthly live roundtable open to the audience, and you are one of the guests at the roundtable where you get to ask questions and steer the panel discussion yourself. So, if you go to DevOpsUnbound.com or check it out on Digital Anarchist, TechStrongGroup.com, or DevOps.com, you’ll be able to see the schedule and maybe join in one of our live roundtables.

    Today, though, is one of our pre-recorded panels—and what a panel we have. Our topic today is low code/no code and many of its aspects and flavors, if you will.

    But before we jump into that, I want to introduce you to our panel today—and I thought we lost one there for a second, but we’re okay. Let me first introduce you to Brian van den Brink. Brian, welcome to DevOps Unbound.

    Brian van den Brink: Thank you, Alan. Yeah, I’m Brian, I work for Valori, it’s a company in the Netherlands. We completely focus on testing, so we don’t have any developers, it’s really all about testing for us, and I worked there for about eight years now. I started, really, as a tester of test automation, et cetera, but then I also noticed in the assignments I did that it was also important to learn something about requirements, how do you get them ready to be tested.

    So, I focused also a few years on business analysis to get going and know what I need for testing, up until, I think, approximately five years ago when I came into contact with OutSystems, and really focusing now on testing solutions specifically for the OutSystems platform and seeing how can you use a low code platform in your benefit as a tester. And yeah, it really opened a lot of opportunities for me, I think, but also for testers, like, how can you work with low code and use it for your testing strategy, so.

    Alan Shimel: Excellent, and thank you very much, Brian. Next, let me introduce you to Derya Sousa, and I’ve been trying all through our prelim to get it right, I hope I did, but if not, say it correctly, but please introduce yourself. Okay, three time’s the charm. I’m sorry—go ahead.

    Derya Sousa: It’s great to be here on this panel today. Thank you so much for having me, first of all. So, I’m Derya Sousa, I’m the Co-founder and COO of a company called Kianda Technologies, based in Ireland, helping companies across the globe. Kianda is a no code/low code application development platform, and we are the enablers of digital transformation and accelerators of digital transformation for companies.

    My background is in IT. I worked as a developer IT consultant across different companies, industries during my career, and I came across the fact of how difficult it is to deliver digital solutions for companies. That’s how the idea of Kianda was born in mid-2016, with the vision of helping people regardless of their technical skills to create the technology they need.

    Alan Shimel: Okay, thank you. Thank you for what you’re doing and thank you for joining us today. Our next panel member is Andrew—excuse me, it’s Andrew Manby, correct? Andrew, if you wouldn’t mind introducing yourself?

    Andrew Manby: Sure. My name’s Andrew Manby. I head up Product Management for our Digital Solutions Portfolio here at HCL Software. You know, in our portfolio, we have both professional developer low code platforms and also citizen developer low code platforms.

    A little bit about me, I’ve been involved in multiple successful software products. Some of them you may know about, a product called DataStage, which we created back in a former company that really, along with Informatica, lit the imagination of the data integration market and have also been involved in other market spaces as well, including bringing thought leadership to IBM around big data analytics.

    Alan Shimel: Excellent. And then, last but not least, not necessary a panel member, but a panel member, but he’s also my Co-host for DevOps Unbound, it’s TechStrong Group CTO and TechStrong Research Principal and Co-founder, Mitchell Ashley. Hey, Mitch, how are you?

    Mitch Ashley: Hey, Alan, always good to be doing a panel together and for the folks on the panel I know, good to see you again. Folks that I’ve met for the first time, like our audience—good to have you all here. I know we’ll have a great discussion.

    Alan Shimel: Absolutely. Thanks. So, guys and gal, let me kick this off. As I was talking about before we went live, I almost view the low code/no code discussion with a BC, Before COVID, time frame, right? Much like ADBC, we have DC and then pre-COVID. I think—and Derya, you’re living proof of this. Low code/no code was alive and well prior to COVID, right? You started in 2016. Low code/no code’s been around, obviously, much longer than that.

    But certainly, with the onset of COVID and the pandemic, never have we seen a need to do more, faster, to get apps out better, to expand the base of people who can develop like we have this past going on almost two years now, right, year and a half—well, more than a year and a half, two years. And so, we’ve seen sort of a rapid development in the state of low code/no code.

    And Derya, I don’t mean to pick on you, but I’m gonna ask you to kinda kick us off, here. I gotta assume that this is what you’ve seen at your company in particular over the last year and a half to two years.

    Derya Sousa: I think to start with, I could tell you, when we founded Kianda, when we would say low code/no code platform, our audience would say, “What do you mean?” Now, we came to a stage that anyone we talk to understands now, most of the people we are talking to, they understand what we do because of this trend and the movement that’s happening with the help of the unfortunate event of the pandemic. But low code/no code was always here, solutions that we had. It started long, long—years ago. You know, we were using Excel sheets, we were using Access, we were using WordPress back then to create our website. So, they are, I think, the initial solutions that we started to use.

    But it came to a stage that the surge of the remote working, the surge of the digitalization needs that are growing really dramatically now in the last two years, it’s going into more enterprises, it’s going into more organization level. And what we are seeing, this is a problem that’s not a new problem. Currently, only 0.5 percent of the work population is capable of coding.

    So, what was happening before was, you know, business users, anyone working in organizations, they were in need for technical solutions, and they were always waiting for IT teams to go in and create these systems for them. But it’s changing now, there is more inclusivity in creating technologies, empowering people, regardless of them knowing how to code or not. It’s giving an amazing world of opportunities for people who were not able to be part of it, be actively part of digital transformation are now becoming what we call citizen developers to create these technology companies, organizations are in need. And of course, we live in a digital first world now with everything that’s going on around us. Any company who doesn’t try to benefit leverage from these technologies that are rapid application development are gonna stay, unfortunately, behind when it comes to competitive advantages and improving or creating that economical value for the organizations.

    Alan Shimel: Agreed. Folks on my panel—thoughts, follow on to that?

    Andrew Manby: Yeah, I have a few thoughts. So, you know, as Mitch and I have discussed a couple of times, right—for me, low code or sort of no code has been around for a long period of time. In fact, my father-in-law was programming in dBase, you know, many years ago, right? [Laughter]

    Alan Shimel: That’s another name I haven’t heard in a while.

    Andrew Manby: Yeah, okay, let’s get Delphi, FoxPro, all of those things out on the table. Let’s do it.

    Mitch Ashley: I think I met your father-in-law, so. [Laughter] [Cross talk] dBase.

    Andrew Manby: [Laughter] Yeah. So, all of these things have been around for a long period of time, right? And if technology has taught us anything is, they evolve to serve a particular need and that’s because the business was trying to accomplish a particular outcome, right? So, they built something, they’ve effectively had it deployed.

    Now, what’s happened now is, you know, we have a lot more tech savvy people in the workforce and, you know, IT is taking a little bit more of a proactive approach. They want a bit more governance, they want a bit more standardization, they also want to be assured of the security and governance of what the business buys, and I think this generation of low code and also no code tools, which I think are two separate things, by the way—

    Alan Shimel: Absolutely.

    Andrew Manby: – help us to do that. And so, you know, I don’t think it’s new. You know, what’s actually interesting—and I hope we’re gonna get to this, Alan—is the type of things that we’re seeing that people are building with these individual products.

    So, I think it was always there beforehand. I think a lot of organizations like, for instance, we have one major airline and there’s a couple of hundred people who are business users building applications on it, right? But typically, you know, to be clear, a lot of them are employee-facing, and they fill the gap between some of those traditional enterprise applications or maybe to some of those SaaS franchises that we often hear about now to get the job done, right? So, I think it was alive and well before, and I think it’s continuing.

    Alan Shimel: Absolutely, it’s continuing, but I think it’s changing. And Brian, I think you might serve as a poster child.

    Brian van den Brink: I think, actually, if you look a few years ago, like maybe four or five years ago, a lot of companies were maybe starting to work with, for example, health systems with low code platforms, using them as smaller apps, basically, in their organization.

    For example, you know, logging hours of your work, whatever, and then starting to grow, like—hey, we can really get to a product really fast, and then other departments of the organization see, hey, they realized an app in a few months, and it’s interesting to maybe expand on it. And then you see, like, a center of excellence within a company start to build apps, and then it grows bigger, and now, I think, currently more and more organizations actually use low code for their core applications as well. So, actually, the most important processes are also done with low code nowadays, and it’s growing like that, because they see the benefits, I guess. And when I see—

    Alan Shimel: Well, why shouldn’t they?

    Brian van den Brink: – yeah. And what I see in my work as well is that, if you work in a high code environment, it might be difficult for a tester or for a business analyst to say, “Hey, this is how it works” and dive into the product, you know, dive into the code, then you need these technical skills. But I think with low code, it’s easier, for example, for a tester to say, “Is this really what we meant with this piece of software?” and you can really get the dialogue going with your developer, even with your product owner, for example, to see, is it really doing what we want it to do, and how can we improve it, for example? And that’s so much easier in a low code environment, I think, than in a high code environment.

    Mitch Ashley: You know, Alan, there’s some—Accelerated Strategies, now TechStrong Research did some research earlier this year on low code/no code and there were some interesting trends in that. And I think part of it was driven by, Derya was talking about the effect of the pandemic; you were, as well. That rose software to prominence in terms of business strategy must-have capabilities. And along with that, sort of low code/no code was sort of viewed by IT as something that happened outside of IT, we don’t really care about it, the business units are doing something, whatever, to it is a mainstream platform.

    And I’m not saying it wasn’t before, but I think the embracing of low code/no code as a strategy, I think that improved over the last 18 months, and you can also see it by the need to access data, what information people need to get to, because no application is truly an island. They eventually have to get security, they eventually have to get to data and other systems and the fact that, Andrew, you talk about citizen developers and professional developers both as IT viewing those as legitimate creators of applications.

    Are you seeing that trend as well? That certainly bore out in our research.

    Andrew Manby: Yeah, so, we have some of our own survey that’s coming out very shortly that we did with Forrester. You know, we actually found that, from a business decision making perspective for the people surveyed, 80 percent of them said that low code was one of the top three IT priorities for the organization, right?

    Mitch Ashley: Mm-hmm.

    Andrew Manby: You know, 77 percent of the respondents actually said that, you know, agility and velocity were the two most important things to their business in achieving that.

    So yeah, I think companies are realizing that. I do think that there’s a couple of other different phenomena, and I think, Mitch, you and I have talked about this when we’ve had a separate conversation. I do think there are the organically grown stuff that Brian was talking about, which is, you know, you provide a citizen development tool, you establish a center of excellence, right, and then you sort of start draining the desire of the organization to fill the gap with building these applications.

    But I think there’s also—and maybe you touched on it, Brian—you know, organizations are taking also a top down type approach, right? And they’re saying, what are the important digital touch points I have in my organization and, given the advent of COVID, what do I need to be able to fill the gap or continue my business, right? So, if I’m a retailer, through the early part of COVID, one of the biggest challenges I had was, how do I handle, you know, touchless commerce? How do I pick up the deliveries and still allow my retail department to function, right?

    Or say, for instance, I’m a tax office, right, and all of a sudden, you know, the tax function, there’s no people in the office, and what we want people to do is start filing an abeyance request to delay the delivery of the taxes—how are you gonna do that? Or, you know, families are looking for special support, family services support, right? So, those types of things have suddenly become very, very important and low code, whether it was built by the IT organization or it was built by the business has really helped to fill the void. Sometimes, there have been brand new things, and sometimes they’re just really about adapting the business to the new reality.

    Alan Shimel: Absolutely. Let me weigh in, here. You know, I think prior to the pandemic, there is a long history of low code as I think, Andrew, you pointed out, right, though it may not have been called low code then. But prior to the pandemic, low code/no code was around, but a lot of people viewed it as, “Look, if I’ve gotta get something done simply, it’s Minecraft, you know, for adults.” It’s easy, you know, for simple—as I think, Brian, you mentioned, it’s good if I wanna track my hours or something like that. It’s a glorified spreadsheet.

    And then I think what we’ve seen with the pandemic is, in my mind, we’ve seen three branches of low code/no code, and I think, Andrew and Mitchell, you both mentioned it. One is no code, and I gotta be really honest with you—I got a problem with that. I have the same problem with no code that I have with no ops. I think that’s pure marketing, and I don’t know what you can truly, truly do with just no code. And we can talk about that, feel free when I’m done if you guys wanna jump.

    Andrew Manby: I’m just thinking about your Excel spreadsheet, right?

    Alan Shimel: Uh huh.

    Andrew Manby: You know, Excel is no code until you type in that formula.

    Alan Shimel: Right, and then all of a sudden, you better—take it from me, I’m illiterate on Excel formulas, so I know there’s code there.

    Mitch Ashley: Because you put in an expression, you’re now coding, right?

    Andrew Manby: Yeah, exactly. [Laughter]

    Alan Shimel: But when we look at quote-unquote low code, here is the big takeaway with COVID for me is, we have low code for developer teams—and Brian, I’m gonna include testers in there, because testers were already scripting and doing some coding in order to write their test.

    So, we have low code for development teams—DevOps teams, whatever you wanna call them, developers—and then we have low code for the quote-unquote citizen developer. These are people who traditionally didn’t do any development other than maybe spreadsheets. And, you know, lo and behold, the business analyst, the person over in Sales or Marketing or in HR, they now have the capability of not having to wait or go through that whole IT process driven formula, but they could use some paint by numbers and relatively easily build a very functional application. Don’t equate citizen developer with simple app. There’s some sophisticated stuff we can do now.

    Brian van den Brink: Alan, I wanted to react to that is that, indeed, the citizen developers, but also the IT development teams working with low code.

    Alan Shimel: Yep.

    Brian van den Brink: But what I do see is that more and more companies choose low code to basically restructure their, how do you say, legacy applications, you know, rebuild it or rethink their software. And I think it’s really crucial to still have—maybe you can work with citizen developers as part of, okay, what are the business requirements we need and how are we gonna implement it, et cetera. But it’s super important, I think, to still have your development skills on par with what you need. Because even with low code, maybe it’s easier to make something, but you still need the same kind of skill set that the developer would have to know how do I structure things, you know, how do I set up my architecture around it. Those kinds of things are still development skills, maybe without coding, but it’s still development skills, I think.

    And that’s why I think you always need maybe a senior developer, whatever, in your team to be able to handle those kinds of questions and then supported, I think, of course, by juniors or citizen developers even, but the development skills that you need to build an application are still very much necessary, I think.

    Alan Shimel: Agreed. Derya, it looks like you wanted to—

    Derya Sousa: I would like to add something, a point to what you said, Alan, to your definitions. So, there are solutions in the market that low code solutions, they generate code for a developer to go in and speed up this process of delivery and make the changes as they receive them.

    Alan Shimel: Yeah.

    Derya Sousa: There are solutions like what we provide. We don’t generate code, so we call ourselves as our customers call a true no code platform. However, we allow developers to go in and create widgets, create rules and actions that can help them bring that end to end business application to life. And I think that’s important, because there is a good bit of differentiation between them, and from our experience, we work with IT teams as well as Business Departments.

    Alan Shimel: Yeah.

    Derya Sousa: But we start with IT Departments, because they’ve seen the benefits they can get—

    Alan Shimel: The strength, the power.

    Derya Sousa: – by using a platform like this—exactly, filling the gaps that they might have. And now, even since the pandemic, we are talking to more public entities for—public hospitals, for example, for government agencies. Even them, they tend to be slower to catch up with innovation and technologies. Even them are looking at low code/no code systems, how they can use their legacy systems, which they’ve been using maybe two decades, but they had no idea how, actually, they could bring a user-friendly interface on top of these systems.

    So, that’s where, I think, solutions like rapid application development solutions that come into place and bring a lot of benefits to organizations.

    Andrew Manby: Yeah, I would agree that certainly, you know, some of those you think are traditional sort of holdouts, right, typically on the laggards in the adoption curve are certainly doing that. You know, we’ve seen that in state government, we’ve seen that in large tax authorities. We’ve also seen that in the District of Columbia Motor Vehicle Registry. But I think also one of the things that I want to make sure that we add to the conversation, I sort of touched on it is, you know, I think organizations, I’m just thinking about actually the first private distribution only utility in North America is the Vermont Electric Company. And they, a lot of companies want to go through additional transformation, right? I know we’ve all heard that term, right? But they really set out a plan for a number of years to sort of think about how they could not only replace some of the legacy, but really sort of think about, you know, shifting the cheese on what does it mean to digitize the processes that they actually do.

    So, I think, for some of the organizations, you know, they’re not only thinking about—okay, how do I put a new front end on an existing system where, you know, it’s been particularly inflexible. But I think also, a lot of them are a lot more, you know, thinking about how do I, you know, move the cheese? So, in the case of the D.C. DMV, how do I, for instance, say, “I don’t want to have people go into a motor vehicle registry to register their vehicle, right? I don’t need them to go into a public office to pay a parking fine or a speeding ticket—what if we just actually changed our processes and put it on one of these things, right?

    So, I think a lot of organizations out there are actually thinking about low code in that way, and I think that’s where the conversation shifts from being an IT way and an enabler to also being—alright, you know, the CMO, the COO has a seat at the table and she is saying, “Look, how can we improve the engagement with our customers where our traditional ways of engagement with them have change significantly?” And I see probably equal amounts of companies coming to us and saying, “I’d like to do this,” right? You know, “I want to change the way people buy tires for their cars,” right?

    Mitch Ashley: You know, in thinking about this, it seems that sometimes the terms we create don’t always serve us well, right?

    Andrew Manby: Right, right.

    Mitch Ashley: And, you know, visual programming is what we would’ve called low code maybe 5 or 10 years ago and you know, is a macro low code or no code? And actually, I don’t really care, it’s a macro, it’s just like—you know, is Python a scripting language or a programming language? I really don’t care. It sort of doesn’t matter. I think the end result is, you know, you talked about, Derya, the rapid application development, and I don’t mean the RAD term, but creating applications quickly and expanding the universe of people who can do that across the business is all good. I mean, it’s to the benefit, to the end of the business. And we can create a mess for ourselves, or we can do it well, in a way that it works together.

    But I think, hopefully, we get to a point in a year or two where we don’t think about low code/no code, right, it’s just—it’s another way of, it’s an application platform for creating apps. And anybody can do it, because it’s—I mean, whether it’s no code, I don’t care if Trello’s a no code or a low code. I don’t care if Zapier is a low code or a no code, you know, just to pick a few commonly used things. I think we’re about the result we wanna get.

    Andrew Manby: Yeah, I agree, Mitch. I mean, I think at the end of the day, you know, we obviously know that technology goes in cycles. You know, certainly, in our portfolio, we have low code that we would sell to a customer for doing that specific task, right, and those are broad sort of use cases.

    But, you know, low code or that sort of no code citizen developer construct exists in our digital experience platform and it also exists in some technology we have specifically for the DevOps process, which is a low code version way of them to help orchestrate what they do in the DevOps process. And you think to yourself—well, are they a commercial low code tool? Well, no, but they’re bringing the, enshrining the value of a no code/low code tool to a specific challenge.

    You know, maybe that will be the way that this market will evolve over time, but definitely, I would say that where we are right now is, it’s permeating into lots of different places, right? You know, does that mean that, say, is Monday a no code tool? But probably what makes it stickier is the fact that it has some no code type capabilities to it, right? And maybe that’s okay, right, for customizing certain types of applications.

    Mitch Ashley: Well, it ventures into the whole workflow space, right? I mean, you know, Atlassian now is positioning themselves as a competitor to Monday and others as a workflow tool, and you could say that’s low code or no code, too. So, it’s really, when we talk about that as a segment of the market, it really is much bigger than just what we might narrowly define as low code.

    Alan Shimel: Yeah. So, to me, the definition goes back to something Brian was talking about earlier, and that is—what’s the involvement of IT, right? Even with this citizen developer thing, I see it bifurcating. There are some cases where traditional IT teams are calling in a non-traditional IT person to help with the building, architecting, sizing of an app. And that non-traditional IT person benefits from the team being, using low code/no code tools, right? So, that’s one scenario with the IT Department’s involvement.

    Then there’s sort of a shadow low code/no code development kinda operation without IT, right? Where—hey, man, it takes too long, I don’t wanna go through the whole change management process and the architecting, I just need this quickly. And I got the tool to do it and it’s cheap and it’s SaaS an it’s boom, boom, boom and I’m done. “Hey, where’d that come from?” “Eh, I built it.” “You built it?” “Yeah, I built it.” Right? And that’s powerful. I’m not gonna—let’s not kid ourselves. That’s a powerful enabler in an organization when the IT person says, “Hey, wait a second, where’d you get that?” “I built that.” Right?

    How do those two jibe? And I think that’s really where the friction, where the rubber meets the road, and what is low code, what is no code, who does it? I think the delineation is, is IT involved or not? And, you know, Derya, you were saying most of your engagements are going through the IT Department. Brian, you seem to be on the IT Department is involved side of the house as well. Andrew, I’ve heard you go both ways here a little bit. But, to me, that’s the key sort of indicator.

    Andrew Manby: Well, and Alan, can I ask you a question, then, right?

    Alan Shimel: Sure.

    Andrew Manby: You know, I tend to think about low code for citizen developers to be a bit like a sandbox, right? You know, I live—where I live, there’s a zoo a mile away, right, and there’s an area there and a kids’ playground, there’s a tent and there’s a sandbox underneath it, right? And it makes it very easy for the family to sit down and observe their kids, right?

    So, you know, I think the way that no code is developing from an IT perspective, it’s trust but verify. Can I see what they’re doing? If they get into trouble, I can help them out, or as Derya said, you know, if there’s something that they need, right, maybe they need a specific widget to do a payment or they need something to deliver a package or they need some very specific type of math problem and they build an algorithm in the math lab and they want to bring it in, that’s when IT does that, but also, again, it’s that oversight and governance, right? Trust but verify, but you know, who’s using the application? What data do they have access to? Are they doing Select Star from HR executive salaries, right? Is someone curating those data sources for them so they can do no harm in their developments?

    So, I think that’s the place where citizen development is going. I think that’s where organizations want to invest, that’s where they’ll put the COEs. I think the days of, say, my father-in-law building something in dBase will go away from an IT perspective.

    Alan Shimel: So, let me use your sandbox analogy. I’ll put forth the proposition to you that most parents, a lot of mommies, take their kids to the sandbox and leave them there because it’s a relatively safe environment and they can concentrate on other things, maybe doing some of their own work or speaking to the other moms or what have you, and they consider it a safe environment to leave their kids, and they only keep half an eye on them versus, you know, being on swings or something. Same thing, here.

    Andrew Manby: Exactly, Alan. So, I think, for me, you hit the nail on the head, right? If it’s about IT backlog, you’ve effectively given one strategy to halve or reduce your IT backlog because, with your supervision, right, the moms can go off and do something else which they focus on which really requires their skills.

    Alan Shimel: Mm-hmm.

    Andrew Manby: But then, you know, as citizen developers in the sandbox, I can get what I need done for the business faster. Isn’t it like a division of—

    Alan Shimel: Well, let’s go try it.

    Andrew Manby: – a division of labor?

    Alan Shimel: It is sort of a division of labor because we’re not talking about kids in sandboxes. But when you have testers doing low code/no code, you know, building quick, being able to testing faster, better—that, to me, is a very different low code/no code use, right? That’s a—

    Brian van den Brink: What we often see, for example, with Tricentis, Tricentis is a low code testing tool, and I think it’s important as a company if, for example, I choose low code development for my core applications. Then also, you have to probably think about what does that mean for the other roles within the IT Department like, do we have to think about then, also, a low code testing tool that fits our development platform, for example? Often, I think what I still see happening is that we choose low code for several reasons and the decisions that help that. Also, you can rely on—okay, that’s the same decisions we need for our test tooling, for example. And I think what’s important is that I think what the low code opens is that you can work with a low code platform and use those capabilities for a testing perspective as well.

    So, for example, if I test a process, it takes a lot of time to go through all the screens and to test a workflow, for example. And I’m testing a workflow and there’s all kinds of data I need to test specific outcomes of that workflow, try to find solutions in your platform to basically mock that data or to set that data up through a faster way than to going through all those screens, maybe you can use an API or you can leverage something out of the platform to make that faster. And I think that’s where low code testing comes in place is that it’s easier to shift left towards a developer and discuss those kinds of things. Like—okay, we have a requirement we want from this business perspective, what do we need to build that as a developer and what do we need as a tester to test that more efficiently? And I think that’s where the thing comes together is that if we discuss that within a team and then it’s really an IT thing, what do we need from a technology perspective to make everybody’s life easier, basically?

    And I think low code, companies use low code, for example, because they have a challenge in finding the technical people to be able to build their apps. And we have similar challenges in testing that it’s difficult to find the technical people to build the technical scripts. So, there, I think we need to make sure, as a company, that you match these challenges in the same way that—okay, low code platforms probably also need low code testers, because you have, you know, you make those decisions for a reason, I guess. And that’s what I see in my work.

    Yeah, it opens a lot of opportunities as a tester to be more efficient in your work and to make sure that you keep up the pace with your developers. Because low code development is a lot faster than traditional development, and if you don’t change as a tester as well, you will always be, like, a bottleneck in the process, because you’re not as fast anymore. And then that’s why you need to think of smarter solutions as well.

    Alan Shimel: Derya, low code apps need low code testers?

    Derya Sousa: Sorry?

    Alan Shimel: Brian said something that if you have low code apps, it makes sense to have low code testers.

    Derya Sousa: Low code testers—yeah, yeah, absolutely. Just to add to the original conversation about governance and security, I would like to maybe add a couple of points there. I think, again, the solution to what you mentioned, Alan, there is—yes, we deal with IT Departments and, you know, business users are being exposed to these solutions, low code/no code solutions—where is the security, who has access to what, and how we can implement governance. And it sounds like a broad concept, but you know, any organization would need to implement governance, you know, how they operate, what is moss, what is a shoot, what is a can for them to do and now to do things.

    And the same approach should be implemented when it comes to employing this type of low code/no code solutions. So, just because the platform is like a LEGO building block or sandbox, as you mentioned, it doesn’t mean that we should forget about everything, it’s too exciting, let’s build whatever we need and, you know, roll it out quickly. I think the same approach as the company would have had in their change management, change control, governance, they all should be now considered the same way for low code/no code approaches, whether it’s IT building it or whether it’s business users building it.

    And what we usually see that’s working is when IT and Business Departments work together. So, IT creates some kind of a controlled environment for these non-IT users to go in and create solutions while IT is shadowing them or supervising them, giving them tools they need. But at the end, it’s all a controlled environment where, I’m really the biggest fan of giving people free, innovative solutions that they can freely innovate and create solutions they need, but this should not come without a plan. So, a low code/no code system should not be implemented without a plan.

    So, that’s why we are seeing sometimes a struggle that companies go through, because they see the opportunities, they start with one solution and they want to implement so many more in a couple of months’ period, and it can be challenging if there is no plan. It can be challenging if there is no strategy, overarching strategy that looks to different kind of aspects of how to implement such a rapid application development system, yeah.

    Brian van den Brink: Yeah, when we have—

    Andrew Manby: I probably disagree, because I think there’s different modalities. I would say a lot of citizen developer low code is an either they develop it to prototype it and then hand it to IT once they’ve actually proven it, sometimes they’re there for a period of time so it’s literally like a pop-up booth or a food truck. You know, it arrives, it serves it purpose, and then it’s torn down, right? So, I don’t think you can necessarily have IT be involved in all of them.

    I mean, certainly, you have some sort of, you know, diamond team to do that, but I don’t think it all happens like that. I think there’s different ways that, yeah, some of it is definitely planned out through IT and Business to deliver something. Sometimes it’s a cross functional team and it’s driven by the business. But I think part of the attraction of low code, and we mustn’t forget about it is, it’s that organic nature and the empowerment of the people in the business to get their job done.

    And sometimes, you know, back to Alan’s point, it’s—you know, you don’t wanna get slowed down by what IT’s doing and as vendors, if we do our job correctly, we give them that safe space to be able to do that, right? Trust but verify.

    Brian van den Brink: I think it’s important to—I think, indeed, what it opens, and I agree there is, it allows you to prototype things from a business perspective, but I think what’s important then in the next space is that that prototype is not to go in our production environment, basically. So, you know, you have to make sure that sometimes what we say is that, if you look at old spaghetti code, with low code, it’s easy to make, like, fast-cooking spaghetti, basically. Because, you know, you can make your mistakes faster because it allows you to create things faster, as well.

    So, I think, indeed, you start with a prototype, you verify the basic concept, and then at some point, if you really wanna structurally have a core application out of it, it has to go through architecture principles and things like that. But I think that’s what Derya also means, maybe, with the plan is that, once it goes to that, after that first phase, it really goes into that planning phase and how are we gonna structurally broaden this application, for example?

    Derya Sousa: Absolutely. Just to add there, so, the applications we build, we deliver, or our customers are delivering are core applications, core business applications, and I don’t see them working well in terms of security and governance if they are just let and there’s no plan or there’s no cooperation with IT teams.

    So, from my experience, I’ve seen this and there are, really critical solutions are being built with low code/no code systems, and they are connecting to critical data as well, and especially with security in mind, that we all have to have nowadays. I don’t see this getting to the highest potential without IT being involved.

    Andrew Manby: Yeah, I will agree about that, but I would argue, continue to argue that there are a lot of things that need to be put in production in a safe space that have been curated by IT that we just allow the business to run with, right? That’s just the nature of the beast, right?

    Alan Shimel: Absolutely

    Andrew Manby: And to be honest, I don’t know why you would want to change that.

    Alan Shimel: Well, this is a great conversation, but we’re out of time, we’re over time. But, you know, obviously, if you couldn’t tell at home listening to this, there’s a lot more we could—we could’ve spent all day talking and still not cover it all. Maybe we’ll have to re-run it and bring it back for another low code/no code episode, soon.

    But we’re gonna need to wrap up this episode of DevOps Unbound. Brian, Andrew, Derya—thank you so much for joining in and contributing to this panel, it was great. Mitchell, as always—good job, man, you know? You’re doing this—

    Mitch Ashley: Enjoyed listening to the panel as well as participating. It’s a great group.

    Alan Shimel: Absolutely. Many thanks to Tricentis for sponsoring DevOps Unbound. We’ll be back in two weeks with a fresh topic related to DevOps. Hope to see you there and have you join us, but for now, this is Alan Shimel for DevOps Unbound. Have a great day, everyone.

    [End of Audio]

  • Oxeye Platform Helps Fix Code Vulnerabilities

    Oxeye Platform Helps Fix Code Vulnerabilities

    Fresh from raising $5.3 million in seed funding, Oxeye emerged this week from stealth to launch a namesake application security testing platform that, in addition to pinpointing issues in code, also provides advice to best remediate the issues it found.

    The Oxeye platform requires developers to download an observer tool that scans code for vulnerabilities. Once pinpointed, the Oxeye platform guides developers through the remediation process without requiring them to inject agent software into every application.

    Oxeye CEO Dean Agron noted developers don’t typically have a lot of cybersecurity expertise. As application environments become more complex—thanks to the rise of microservices-based applications—it only becomes more likely that mistakes will be made. The only way to resolve security issues before applications are deployed in a production environment is to augment developers using a platform that finds vulnerabilities all the way down to specific lines of code, he said.

    In theory, responsibility for application security is being pushed further left toward developers as organizations embrace DevSecOps best practices. However, making developers more accountable for security without providing them the tools required to achieve that goal is not going to improve the overall state of application security.

    The challenge organizations face today is finding a way to empower developers to address security issues without slowing down the rate at which applications are built and deployed. The best way to achieve that goal is to give developers access to a security tool designed to both analyze code for vulnerabilities and explain how to avoid encountering that issue again, said Agron.

    Left unaddressed, the current software supply chain crises will only deepen, added Agron. According to International Data Corp. (IDC), more than 500 million digital applications and services will be developed and deployed using cloud-native platforms by 2023. That’s roughly equal to the total number of applications and services deployed over the last 40 years, according to IDC.

    It’s not clear to what degree the rise of digital applications might force the DevSecOps issue within more organizations. It’s is clear that a new generation of more complex applications based on microservices is being rolled out to drive a wide range of mission-critical applications. The more business value an application has the more attention it inevitably attracts from cybercriminals.

    Regardless of the type of application, it’s apparent that there is a pressing need for a way to better secure applications. The further left that process starts the less costly it becomes to achieve and maintain security. Cybersecurity teams have a vested interest in helping developers do the right security thing as early as possible if no other reason than to reduce the number of vulnerabilities they will inevitably discover later.

    Making developers more aware of security issues is, of course, only the first step. However, in the absence of any tools that can be easily incorporated within an application development workflow, the concerns being raised will amount to little more than yet another sermon that will be ignored as developers once again rush to meet yet another application delivery deadline.

  • DevSecOps Progress: Slow and Steady

    DevSecOps Progress: Slow and Steady

    [pdf-embedder url=”https://devopsy.kinsta.cloud/wp-content/uploads/2022/02/devsecops-progress-slow-and-steady.pdf”]

  • Why DataOps is Critical for Digital Transformation

    Why DataOps is Critical for Digital Transformation

    Data lies at the heart of all digital transformation. In fact, today, as organizations look to further accelerate their ongoing digital transformation initiatives, companies are looking to extract even more value from their data, to better drive corporate decision-making, help grow the business, improve customer satisfaction and gain a competitive advantage. 

    Inspired by DevOps, DataOps, which Gartner defines as “a collaborative data management practice focused on improving the communication, integration and automation of data flows between data managers and data consumers across an organization”, has grown in popularity in the past decade as organizations seek to accelerate the delivery of business value from their data. But taken even one step further, enhanced data operations (i.e. more than DataOps) can help organizations maximize the business value of the data they own and also the infrastructure in which the data resides to increase data quality, reliability and availability in a hybrid cloud environment. This enables organizations to further improve business resilience and digitalization through collaborative, data-centric operations in an increasingly complex, interconnected world. Particularly as hybrid and multi-cloud environments proliferate and it becomes harder for teams to understand, analyze and streamline data usage.  

    DataOps Enables Digitalization

    By 2023, IDC predicts that 52% of worldwide GDP will come from products and services delivered by digitally-transformed enterprises—and according to Gartner, 91% of enterprises are already involved in some sort of digital initiative. This means that organizations already engaged in digitization are paving the way for a new future of business—commerce, production and innovation. But digitization is only half the battle—the other half (i.e., digitalization) comes from investing in understanding those digitally-driven insights and using those learnings to fuel business growth. 

    In the face of stifling market competition and evolving customer demands, the key differentiator in market leadership today is data and operationalizing that data. Hence, the difference between the market leaders and followers boils down to this: Data operations. Data operations not only plays a vital role in empowering ongoing digital transformation initiatives (think hybrid cloud operations, DevOps optimization and application modernization; data is at the core of all of these efforts) but also serves as a foundation for faster, better-informed innovation—fueling organizational growth and propelling businesses ahead of their competition.

    Who is Driving Digitalization?

    To gauge where your organization stands on your digitalization or digital transformation roadmap, you must first identify who within your organization is driving it and who’s involved. The latter is an easy enough guess—because, essentially, it’s everyone. 

    Although digital transformation is defined as the transformation of the digital operations of an organization, it reaches far beyond IT to all major functional departments and requires a cultural change to reach its fullest potential. That includes everyone from the chief information officer (CIO) and chief data officer (CDO)—collaborating with key stakeholders to map out and define digital transformation strategies and efforts and communicating goals and objectives—to the technical and professional teams—working closely with business units to design and operationalize new architectures and technologies to advance digital transformation initiatives. At the core of all of these modernization efforts is data; the metric for business success in the digital age. As such, integrating DataOps into your digital transformation efforts from the start is mission-critical. 

    As the market becomes increasingly competitive and business leaders look for new ways to grow and expand their business, data will remain a key catalyst in achieving those lofty digital transformation goals. By gaining intelligence from their data, companies can realize the ultimate goal of data empowerment—where all employees have simplified, secure levels of access to the high-quality data they need to do their jobs—enhancing operational efficiency, improving business outcomes and fueling business transformation at scale. 

  • The Power and Potential of Stored Procedures

    The Power and Potential of Stored Procedures

    One way of saving time when writing code is with stored procedures; prepared SQL code that you can save so the code can be reused over and over again. Stored procedures are a powerful vehicle for speeding up your code and giving you gains in performance. 

    Stored procedures have many benefits—they can be easily modified, are reusable and they possess the ability to automate a task that requires multiple SQL statements. Another advantage of stored procedures is the ability to access or modify data in a database.

    Before incorporating stored procedures, it’s important to know exactly what a stored procedure is, how it differentiates itself, how to create a stored procedure and how to use them in embedded analytics.

    Performing Tasks With Stored Procedures

    A stored procedure is defined as a set of SQL statements and other PL/SQL constructs that are stored in a relational database management system as a group that you can save and reuse. A stored procedure can be comprised of multiple SQL statements like SELECT, INSERT, UPDATE or DELETE. These statements run as a unit and are used to solve a specific problem or perform a set of related tasks.

    Stored procedures are among the most frequently used and most popular database subjects. Using stored procedures can help simplify and speed up the execution of SQL queries.

    Stored procedures can reduce network traffic between servers and clients, for example. This is a result of the commands being executed as a single batch of code rather than multiple. Rather than every single line of code being sent for execution individually, only the call to execute the stored procedure is sent to the server.

    The other benefits of using stored procedures include maintainability, performance and efficiency and complex business logic. And maintaining one stored procedure is easier on the server than maintaining copies on multiple machines. Procedure scripts are stored in one location and they can be freely used in other applications, too. In the event of a definition change, only the stored procedure will be affected—not the application that calls it.

    Advantages of Stored Procedures

    Once you compile a stored procedure, it can then be used many times. This can save a lot of time writing, especially if you are doing ad-hoc queries. This also gives you a slight performance edge on the first execution of a stored procedure over the first execution of an ad-hoc query.

    A stored procedure is ultimately a function call on a database. You may include IF statements, LOOPS, security checks and more to ensure the correct data is being sent back to the call of the stored procedure.

    Another advantage of stored procedures is that they are easily modified. Simply change one or more statements in a stored procedure with the ALTER PROCEDURE command and it quickly updates the results sent back to the calling app. Unlike a middle-tier component which includes business logic that would need a potentially costly re-deployment, a quick change to a stored procedure takes seconds.

    Another benefit of using stored procedures is the security it provides to its data source. The security of an application can be enhanced when using stored procedures by restricting unauthorized access. You may grant someone access to a procedure that updates the database table without allowing access to the table itself. You may also use stored procedures to “hide” the underlying table names from the end user of the stored procedure. Since the caller of the stored procedure has the name of the stored procedure and not the name of the tables used in the stored procedure, they cannot directly access the tables with your data.

    Best Practices For Creating a Stored Procedure

    Creating a new stored procedure is fairly straightforward. Here are the three basic steps:

    1. Within the SQL Server Object Explorer, right-click on the Stored Procedures node under Programmability.

    2. Within the New Stored Procedure Template, modify the CREATE PROCEDURE statement to include the tables, parameters and query options to return the desired data from the stored procedure.

    3. Within the Toolbar, click the Execute button to compile and save your stored procedure.

    There are multiple tools you can use to create stored procedures. Using the interactive SQL Server Management Studio is the easiest, as it gives you an easy way to start connecting to many different types of database servers, including Azure and SQL Server Analysis Services.

    Behind the Scenes in Embedded Analytics

    More organizations are using data to predict future outcomes and stay competitive, offering their customers what they want when and where they want.

    Embedded analytics business intelligence software holds the key to unlocking the power of data. Stored procedures can be incorporated into embedded predictive models and continuous computational language code for data streaming scenarios.

    To create a new visualization on a database that supports stored procedures, first review the list of stored procedures that are available. You will treat the stored procedure you select just like any other data source. The only difference is a parameterized stored procedure, which you can use in a Filter by passing variables to the stored procedure.

    Some embedded analytics platforms, for example, offer complete programmatic control over stored procedures and parameters via code. To pass a stored procedure name, as well as any required parameters, use the ChangeVisualizationDataSourceItemAsync function. This is an example of passing a stored procedure named CustOrderHist and passing the CustomerID parameter in the Reveal platform.

    The Power of Stored Procedures

    Stored procedures are a powerful tool to query a database and return results to a calling app. They are secure, offer performance advantages and are easy to maintain and modify. Stored procedures can be a good avenue for simplifying the way you present data and deal with parameters in your application code versus using error-prone and lengthy ad-hoc queries. As you develop applications, you may want to consider incorporating useful stored procedures in your next project or visualization database.

  • Open Source is the Engine for Observability at Scale – Observability at Scale TV EP 2

    Open Source is the Engine for Observability at Scale – Observability at Scale TV EP 2

    As today’s organizations seek advanced monitoring and observability to ensure the performance of their microservices and cloud native systems, issues of cost, complexity and tool sprawl have emerged as pervasive challenges. Join our hosts Alan Shimel and Mitch Ashley along with panelists Dotan Horovits (Logz) and Constance Caramanolis (Splunk) as they dive into the challenges that DevOps practitioners are frequently forced to compromise on and how they transform modern observability. The video and a transcript of the conversation are below. 

    Alan Shimel: Hey everyone, welcome to Observability At Scale, one of the latest video series or video show series, over here on TechStrong TV part of TechStrong Group. I’m your host, Alan Shimel. And let me just say that Observability At Scale is sponsored by our good friends at Logz.io, it’s L-O-G-Z and many thanks to them for sponsoring this. This is a great subject.  

    I recently in KubeCon about two weeks ago. And I will tell you, it was pretty much all Observability all the time. I did 40 interviews there in two days. And I wouldn’t be surprised if somewhere between a third and 40% of the people I interviewed were about Observability. And many of those being at KubeCon CNCF Cloud Native Con, many of them were about open-source Observability. 

    Whether we’re talking about Prometheus or OpenTelemetry or several of the other Observability-related open-source projects out there. That seems to be the subject of today’s episode: Observability and open source. And I think we’ve got a great panel to discuss this. Let me introduce you to them. First of all, we have Dotan Horovits. Dotan is with Logz. Actually Dotan, you introduce yourself. I don’t want to put words in your mouth. Go ahead.

    Dotan Horovits: Gladly. Glad to be here, Alan. And my name is Dotan Horovits, and I’m a developer advocate at Logz.io. Logz is a cloud-native platform for observability. And the nice thing that I love about Logz.io is that it’s all based on popular open-source tools that we’re going to talk about extensively today like the ELK Stack and Prometheus and OpenTelemetry and Jaeger and so on. So that’s about Logz and myself.

    Alan Shimel: Thank you, Dotan. Next up, we have Constance Caramanolis, and I hope I did that justice. Hey, Constance, how are you? Maybe a little bit about yourself for our audience?

    Constance C.: Good morning. I’m good. Hi, everyone. I’m Constance. I’m a principal software engineer at Splunk. Right now I’m on OpenTelemetry Governance Committee for I guess another five days until elections are over. But I’ve been involved in open source for about four or five years now, both on Envoy OpenTelemetry and on the product with Con community.

    Alan Shimel: Excellent. Thank you, Constance, for coming on today. We appreciate you having you here. And then last but not least, is my co-host of Observability At Scale. He’s also CTO here at TechStrong Group and a principal analyst at TechStrong Research. It’s Mitchell Ashley. Hey, Mitchell, how are you?

    Mitchell Ashley: Good, good, good. Great to be here and very thankful to Dotan and Constance for joining us. I look forward to a great conversation.

    Alan Shimel: Yep. Thank you. So guys, as I started the show off I was out in Los Angeles, I guess it was two weeks ago now for KubeCon Cloud Native Con. And first of all Linux Foundation did an amazing job creating as safe an environment as possible for us to do an in-person event. And it was great to be in-person they got. I don’t want to belabor it but this stuff is good on Zoom, it is better in-person. 

    One day maybe we’ll run this as a live panel from a KubeCon, and that would be just great. But it was all Observability all the time, it seemed. Several things with Observability. Number one, the hairs on the back of my neck stand up when I go to a major show. And there’s something that’s so hot, so hot, that I just feel I can never live up to its promise. And that is what I was afraid with you all. 

    Are we in a overhype cycle around Observability. And then secondly, so much of Observability is based on open-source underpinnings, if you will. Right. There’s a lot of great open-source programs. Maybe that’ll save us from the usual hype cycle that we get here, I don’t know. Anyway, I’m interested in your comments on that, is Observability being overhyped at this point?

    Dotan Horovits: If I may take the first pass on this one. I think that as many trends, and we’ve seen the quite a few of those thought out the years, you have so many vendors trying to piggyback on the trend and leverage that for the relevant offerings that it creates a bit of a blur and confusion about what exactly Observability is about.

    Is it about tooling or methodologies? Is login in its own right for vendors that just provide, let’s say, login of vendors just provide metrics is that Observability? And can you rebrand for Observability yes or no? You have two of these, is that enough? Is signals the whole story about Observability at all? Or is the discussion about Logz metrics traces do justice to Observability?

    I think these are the sorts of conflict and confusion that I see and sense in my conversations with the user community, with customers of ours, with partners of ours. And, again, lucky enough in my position as an evangelist, I use my time to try and clarify and try to separate between the tooling and the methodologies and the essence of things. But definitely, a lot of confusion around that.

    Alan Shimel: A lot of confusion. I did an interview or a panel discussion yesterday with some folks. And one of the things, there was a good line that came out of there was actually around the definition of DevOps, but it’s equally, I think, applicable here, which is, what is Observability? And all of these kinds of things you mentioned, Dotan is, where is the value? 

    Show me where the value is in and I’ll show you what it is, who’s using it, who’s doing it, who’s not. Where is the value? Constance, one might say you were into Observability before it was cool, before it was hyped.

    Constance C.: True. Yeah, right, because that was one of the selling points of Envoy, was making things transparent to the networking to everyone around. And one of the features that made people really love the project was that we would joke, there’s a staff for that. You would send a request, there would be staff for the connection, and everything related to it, and so that made things really easy to debug as a consumer of it. 

    Dotan, you’re definitely right in terms of so much confusion because observably can almost be like the block-chain. And no offense of block-chain, but that’s like the key word that gets thrown around in terms of getting really hip and cool. I think the nice thing about this attention is that it is forcing a lot of people to start thinking more about why they’re doing things.

    There’s a lot of just doing things because you’re either told to or your thought about and now sort of be like, hey, if you’re going to do something, we’ve always asked how do we know at work, but it’s like, oh, let me just write a test. Now it’s a little bit more like how do I systematically know it works by either adding instrumentation so I can consistently monitor it or observe it. 

    I think that is a really good thing that we need to start moving towards, because it’s a little bit more of a, we still check something in, throw it over our shoulders and be like, whoa, good day, everyone. And we’re starting to think a little bit more that sustainability. 

    And I also think there’s a second part too, especially when it comes to these projects that were mentioned earlier, we’re not just thinking about tools anymore, we’re sort of seeing them as products and starting to actually say why this could matter to you, why you should actually adopt them. 

    And that’s something that as engineers, it’s not in our normal toolset in how we think about things, we’re starting to get better at, because we need to start. We’re also selling things for open source, the currencies, commits and contributions and not necessarily money.

    Alan Shimel: I think that’s super important. To the question, is it overhyped? Will it be overhyped? Yes, yes. And whatever the next question is, yes. But I think what Observability has going forward is two big things. One is it started from open sources, it started from a community. Second was trying to solve a real problem around as we move to micro-services’ architecture containers.

    How do you manage this stuff? How do you understand what’s happening? I remember Constance, you’re on a podcast we did together, we were talking about how do you know what happened where, what’s doing what, and what do these things do, to use generic terms, but what is it? So we’ve created this flexible, malleable component containerized architecture. 

    Now how do we understand it? We operate it and improve it over time. I guess the other thing we were talking to, you said software engineers, developers, operations people, they’re not into marketing hype. They’re pretty real folks. They want to know, are you solving a real problem for me or is it a checkbox and you’re calling it monitoring observability?

    So we have a pretty healthy skepticism in the market too that will fair it out, I think in most cases, what’s real, what’s not. Where the value is, to your point, Dotan. Okay, go on.

    Dotan Horovits: If I may just add on that, one of the benefits of the open source tools that we’ll talk about and standards, by the way, it’s not just tools, it’s also standards, such as OpenMetrics and others, is that it’s usually driven by real needs of real companies. Actually, today, I had an episode with a gentleman, senior SRE, site reliability engineer, from Google in charge of the entire identity services package of Google. 

    So when you log into your Google account, it goes through their micro-services. And many of the tooling that they use, and they’ve been using for a couple of decades, is the predecessor of what we currently talk about at the CNCF and many others. So obviously, Kubernetes itself, the containerization is a successor of Borg, their internal containerization orchestration system. 

    And then you have Google sensors that evolved into open sensors that now is emerged into OpenTelemetry together with OpenTracing. And you have the Dapper project that very, very fascinating research paper about distributed tracing that was, I guess, the forefather of Zipkin open source and the Jaeger open source and many others in the industry. 

    The nice thing is that Google’s scale is such that where we are at today was where they used to be a couple of decades ago. So in a way, these are well rooted within realistic problems at scale in micro-service architecture, in containerized environment, cloud native environments and technology. 

    So I feel much more confident basing Observability standards, tools and practices on the open source rather than on some vendor initiatives that again, might be inspired by some strategy and might have some forecasting behind it. But much less solid ground around Observability practices hands on.

    Constance C.: Two things I want to add to that, one is like, I think, I guess this is one of the things I want to keep on advocating for, is that I think we should get more just for nitpicking by cognitive doesn’t actually mean containerization. I think it’s just —

    Alan Shimel: And it doesn’t mean Kubernetes necessarily either.

    Constance C.: Exactly. Yeah, that was actually one of the points I was trying to make is cognitive is like any of the things there. But it’s more just I want us to dump that because a lot of people when they think about, oh, let me modernize my stack, I’m going to go straight to containerization. But that might not be the problem that they actually dissolve. 

    And so that’s just one thing I want everyone to be aware of when people are talking about cloud native is like, what is the problem I need to solve when you’re upgrading your stack, remodeling your house, thinking about that problem there? And I guess, I don’t know, Alan, if you want to talk to us now, but the vendor influence. The vendor influence is really interesting.

    Alan Shimel: I was going to bring that up. Let me jump into that. So let me just also lead off with this stuff. Look, Gartner has been doing a hype cycle for as long as I’ve been in technology, 25, 30 years. And the hype cycle is real. That is the way our industry works. Things get really, really overly high, they do the trial, disillusionment, and then they become productive, or whatever that end stage is. 

    What’s a little different about Observability is it’s in this new foundational era of open source, where you have, as I said, several really good open source projects that are driving so much of the Observability innovation out there. None of those projects are controlled by a single vendor. 

    They’re owned and managed by Linux Foundation and maybe some other foundations, which creates this whole golden era of coopetition, to your point Constance, among all the vendors. And this is a very interesting thing. So everybody starts with that same base. When Mitchell and I were doing vulnerability scanners 20 years ago, you could use the open source that was Nessus.

    Nessus was still open source then. You could use the Nessus scanning engine but there were proprietary scanning engines that people were using, some of them really based on Nessus some not. Not the case here. Here, you got a company like Logz, and Dotan, I don’t mean this in a bad way. But you got a company like Logz, for instance, almost their entire value prop is built upon open source. 

    Built upon that open source. I spoke to the Splunk people, their observability stuff, built upon open source. I’m just made mentioning that because we’ve got the two of you here, but there’s a lot of observability vendors out there. And they’re all built upon the same open source foundational tools. 

    I don’t know if we’ve had this kind of coopetition in a market like this in years past and in times past where, look, everybody starts at that same level and what you do with it. Who makes the best observability solution? I don’t know, I guess it depends on really what your needs are. 

    But it’ll be very interesting to see how in the real world laboratory of market conditions, people all come from the same open-source base, and what the value add is on top of that.

    Dotan Horovits: I would challenge two points there. First of all, we started the discussion from the CNCF and from the Linux Foundation, and we can add on top of that, the Apache foundation and others. But let’s bear in mind that many popular tools out there still are controlled by vendors. We can talk about the ELK Stack, ElasticSearch —.

    We can talk about the Grafana that is highly popular, and it’s controlled by Grafana Lab that actually rebranded itself to Grafana to be synonymous with the open source. So there is another beast out there, open sources that are very popular and very well adopted and very mature, but then again, not controlled by a foundation and this various implications. 

    I guess the most prominent implications are when these open sources are then being relicensed to some degree. So users one day wake up to find that the open source on which they built the entire system around is suddenly being pulled of the open source [crosstalk].

    Alan Shimel: One day we woke up and Nusses was an open source anymore, it wasn’t like that. But that’s exactly the point that I was making with foundational open source and why CNCF and these kinds of foundations —, Apache, whatever, it’s so great. And I’m not picking on Grafana, I happen to like Grafana, I like Loki as well, which is another Grafana project. 

    But the very fact is, if you have one company that’s controlling an open-source project, other companies are very often hesitant to contribute. They may use it for now, but they’re hesitant to contribute. And so in the long run, and I’ve been in open source, Mitchell will tell you, we’ve been in open source 25 plus years. 

    Over the long run the foundational model where you can have coopetition among various vendors prove superior to what I call the big brother open-source model where you have one vendor who owns that, and can pull it at any time or calls the shot. If I’m a VC-funded company, a unicorn or a public entity, and I’m investing 10s of millions, hundreds of millions of dollars, I want to be very careful that I am not dependent on an open-source project that is controlled by a single company. 

    This may be getting off observability and I’m sorry for getting on my soapbox, but I think this goes to the reason why CNCF and that community is so vibrant. It’s so much vitality, why the projects coming out of there are so popular and really changing our game. I look at observability because companies feel like everyone gets dealt the same hand, what you do with it is up to you. 

    No one’s going to pull the rug out from under you. And Constance, you’re on these boards so I’m sure you probably feel that way, but I’m interested in what you all think?

    Constance C.: The vendor has such a dirty word in like CNCF like KubeCon, Cloud Native Con, and open source. Okay, I’m putting on my vendor hat. Well, I’m actually putting on both hats, the vendor hat and also the contributor hat. One thing that the vendors have is that there’s a little bit of superpower. 

    And maybe we don’t as vendors or as leverages, is that we actually get to see a broad spectrum of how things are used. And if we’re doing a good job in terms of like, hey, we’ve seen these things like 10 times over and over so that we think this is really important and bring this to the community that could really help the community. 

    I think sometimes the messaging gets lost, because we’re just like, hey, we need this thing here and not explaining why about there. I think that’s sometimes why vendors get a bad name. It’s such a dirty word within open source, because you feel like, oh, you’re controlling how we want to use things, you’re going to make us buy these things there. 

    If you look at our stats, I can’t remember what the stats are on top of my head, a lot of our talks come from vendor companies because a lot of things we use, they’ve contributed so much time into it. I’m only saying this because I’m trying to make it less of a forbidden word. Because they do have valuable insights, and they do contribute a lot. 

    Then there’s also the end user part who has a lot of different use cases that vendors wouldn’t necessarily see. I also think for like some of the examples you mentioned, it might just be a matter of time before they move over to more CNCF. Because a lot of like, OpenTelemetry is still new, two-ish years. I guess, everything within CNCF are about six, seven years, so really, really new. 

    I think it’s just more like continuation of time, as CNCF is, as you can still see, it’s so new that it’s like iron itself out. It’s starting to be, you could say, the de facto place where a lot of people look to, to donate their projects and be a part of that community. 

    KubeCon and Con keeps getting crazy, crazy numbers of attendees and participation, it’s going to become more and more projects start coming to it, because there’s more of that trust, and it’s a little bit more of a matter of time. I like your, what was it?

    Alan Shimel: Coopetition.

    Constance C.: Coopetition. I’m starting to see it less and less. My scope is OpenTelemetry, I will say I wanted OpenTelemetry be like, we’re all vendors, you’re little skeptical. And as I’m leaving the GC now is like, oh my goodness, just seeing so many different people’s, so many different companies’ perspectives on what customers are targeting, what problem we’re trying to solve.

    And start to view these people at first, I’m like, who are you? It’ll now be like, setting up one on one fairly regularly. Talk about careers and life. We go to open source to collaborate with others who are working on projects that are similar to us or probably they’re similar to us. And this was like another avenue to see the more people and get to know them.

    Dotan Horovits: It’s important to say I work for a vendor — that also contribute to OpenTelemetry and to Jaeger and to others. So definitely I agree with you entirely that the know-how within these companies and the broad perspective of different use cases across different verticals, industries, geographies, gives a very valuable perspective.

    It just requires the right mindset of the vendor, because even if you look at the evolution, specifically around observability, players such as Datadog, New Relic and even Splunk started off with a proprietary agent that did very intelligent things. 

    I agree entirely that the ability with a proprietary agent to fetch the right telemetry, to enrich it with the right metadata to do the right aggregations, to fine tune it to the backend tool, storage and analytics and visualization, that’s perfect. But then again, it is very vendor locked. You can’t correlate across different vendors across different signals, assuming that not the same vendor serves all signals. 

    And that’s what the driver that brought us to OpenTelemetry. And gladly is to say, and by the way, today OpenTelemetry is the second most active project under the CNCF according to the DevStats, CNCF DevStat. It’s really amazing to see. That essentially brought the vendor to their realization that the locking through agents is no longer the future path. 

    And they’re aligning with OpenTelemetry and even contributing know how around the agents back to OpenTelemetry. And now the differentiating factor is within the collector, doing intelligent processing, aggregation and the power of visualization based on that. And that’s the right play, and that’s the play that evens the playing field because everyone gets the same raw telemetry and the raw signals. Now, it is up to you to draw the most value out of that and bring it to the end.

    Alan Shimel: That’s exactly what I was trying to say.

    Mitchell Ashley: With your point, yeah.

    Alan Shimel: That was exactly. I’ll take it one step further. It’s not just the CNCF. We do another video show on here with the open mainframe project of all thanks. Another Linux Foundation project. And it’s really an umbrella for 26 different open mainframe-related projects. It’s the same thing. You got Broadcom and IBM and Rocket and BMC, and many others all working together to be able to do like DevOps on mainframes, project Zoe and stuff.

    The CDF, Continuous Delivery Foundation, same thing. You got all the CD vendors in there.  Yes, there are the Grafanas there. Are the big brother open source players left there? This is the foundational era of all, and mean no offense, why like foundation. 

    But this is the foundational era of open source. You just said, Dotan, that’s it. That’s it. You can’t beat that. And that is why the Splunk and the New Relics and the rest of the world have come around to recognize that as well.

    Mitchell Ashley: Maybe [crosstalk] Alan. May be just kind of fanboy for a minute. What’s really awesome about this conversation is that we’re coming to it from multiple points. It isn’t just about let’s start an open source project, and that grows up into its evolution and becomes popular, etc. That’s one point of entry. There’s other points of entry, the proprietary. 

    We started here, now this has sprung up and we need to, maybe in some cases, companies are forced to. You have to, to be competitive, because something like OpenTelemetry has really become so foundational, to use that word, Alan, that that’s critical. You have to be part of that community, and maybe even have to contribute to be able to do that. I’m referring into a genuine vendor here. 

    And of course, tomorrow, someone else will start. And they’re going to start from where we are today. So what’s cool about this open source, it isn’t one thing, it isn’t one way of starting and one place of starting. You may be, like you were contributing to some early projects when you were at Lyft Constance. 

    Or you may be starting the next company, Dotan, that you’re going to build on OpenTelemetry and maybe one or some other projects, you’re going to join Jaeger, whatever, starts something new. But my point is, it’s not one thread through it, it is multiple threads. 

    But having an underlying activity that is bigger than any one of us, that we are contributing to, that it sort of advances the scrum as we move down the field, so we’re heading in a direction that sort of the rising tide lifts all boats kinds of things, yes, we’re all going to compete. And some people are going to only be interested in sort of their pure open source work. 

    And that’s what I’m going to do. And that’s how kind of a developer I’m going to be or what I’m going to contribute to. Others are going to say, how do I productize? How do I commercialize? How do I build something on top of that? Because I have some great ideas too, to develop it. And that’s what’s cool, how far open source is come from, when Alan and I tried to build some products on earlier open source stuff and to where we are today.

    Dotan Horovits: And that’s the beauty. The beauty is that, many talk about open source in the context of community in the greater good and everything, but then it’s important to say it is also a viable business model. It’s [crosstalk] one that you can build a viable business model, unlike some that claim that they cannot sustain it going forward. 

    And use that as an excuse maybe to take it off the open source or things like that. Actually, just look now, last week, or this week that chromosphere became a unicorn raising 200 million with a one point something billion dollar valuation. And that’s all based on M3DB or M3 that spun out of an internal Uber project, Uber engineering that developed its own time series database, scalable Prometheus, if you’d like, database in terms of time series storage.

    And now it’s a company in its own right with its own open source, in addition to a commercial offering. Now they’re expanding from metrics to also distributed tracing, as far as I understand. So it is happening. It is accelerating, actually, innovation, including business innovation, including new startups. It has sprung up around observability. 

    And it’s a very exciting thing to see that as the open source by polar signals that there is now around a continuous profiling in observability. You see that from all angles, and you see startup and innovation. And I think it’s fueled by and facilitated by the open source. And that’s something that they think is great.

    Alan Shimel: Absolutely. So here for me was the key sort of switch that switched, that enabled this. There was a time when we used to talk about the only open source business model that ever succeeded was Red Hat. That was a common thing you used to hear. And why? Because you look at some of the early open source commercial models, and they were about what we offer support and training. 

    And that was like, how do I monetize open source? You do support and trainings really. Well, that’s not a great model. That’s been proven, not a very lucrative model. Then we started doing some open core, premium modules. So I’m going to give you bare bones and if you want the good stuff, you pay me for it. And that was the model. 

    And look, there’s still companies doing that sort of model, it’s viable. Some people have problems with it, is it really open source? I think it is, but another story. But today, what we have is, Dotan, what you’re talking about. And I think, frankly, SAS has been the great enabler of this. Because now, I can take this open source project, set it up as a SAS app that’s going to do it for you. 

    You know what? People understand that these things cost money to develop, maintain, and deliver. And if I’m getting it as a SAS app like that, and I’m paying monthly or subscription, whatever your subscription is, that’s the way business is done. That’s the way software is done today, whether it’s observability, or anything else in this cloud native world. That has been the enabler.

    Yeah, it’s open source, but I’m getting it as a SAS app, this is what it cost, and I’m good with that. I don’t feel like I’m getting something that I should be getting for free. Ron Gula, who started Tenable Network skewed. I don’t know if you guys know Ron. But Ron always used to say, look, with open source that’s free as in freedom and free as in beer. 

    We’ve concentrated on the free isn’t freedom with observability. It’s not necessarily free as in beer. And I think that’s a key mind view that we need that’s enabled, quite frankly, Logz success. Dotan, right? People say yeah, maybe built on open source, but I’m getting real value out of it. And they’re packaging it all up for me and building on top of that. I’m willing to pay for that.

    Dotan Horovits: I would like to even reiterate what Constance said before because when you look at the vendors, the vendors actually bring value here. The vendors, say, okay, I will make the open source accessible to you. But then again, open source projects, and Constance probably can say a lot about that from Envoy and others, very focused on the core functionality, the core value proposition.

    But for a much more mature organization, it needs an entire wrapping around it that provides DSSO and the authentication, the certifications and all the administrative aspects. The open source developers are not even bothered with that. Even a more powerful UI and an accessibility and user management control agents who are back whatever. 

    And this is the wrapping that vendor with again, vendor that serves enterprises and knows all these needs can augment the open source with these capabilities. I think Splunk is doing it magnificently with enterprises and Logz.io and many others.

    But the idea is that okay, you have the open source that you know and love but the open source beyond having it in a managed version also needs some additional layers to get it to be a commercial grade or enterprise grade. So it can actually be acquired that whatever you run is certified, meets the criteria that you need to run it within your organization at scale. 

    Constance C.: So I guess I’m going to say I’m very disappointed all three of you. I guess you didn’t watch my keynote.

    Alan Shimel: No, I didn’t. I apologize. But tell us.

    Constance C.: Okay, so I don’t necessarily know if it should be offered. Some things shouldn’t be offered as like the vendor implemented for you. But the main thing, one of the points I was trying to make that CNCF is more than Kubernetes. That’s why I was calling out the Cloud Native. We don’t think the — hold it. But the next thing for CNCF you have to keep.

    Actually let’s use it like the hype train. CNCF is on this hype train. And it’s great, because it’s celebrating all this hard work. I’m going to say as a community, we actually deserve quite a bit of attention, because we’ve done a lot of great work. But for us to keep on having celebrations like KubeCon Cloud Native Con, and community based, and just having these fun discussions, we need to make sure that we think of CNCF as a whole product. 

    And each project needs to think as a product. And we actually need to start thinking about integrations. Because as CNCF as a whole, we have great projects here. But there’s no glue between them. And so once you drop one project, sometimes you get some other project for free. But if you’re trying to integrate multiple things at once, you get a hard time.

    You don’t get any freebies once you implement, say Kubernetes and you’re trying to implement Envoy, both things there. No overlap, and so we need to start thinking about from a more together. I don’t honestly know that should be offered to SAS, I actually think that – I know, there’s a bit of debate about should there be CNCF Stack. 

    But I think that we should probably actually have something easily set up 10 projects that were really great together. All look great. And that can help things there. The thing that I really like about where observability space maybe for the value add as vendors, is like the key distinction. For OpenTelemetry, it’s about instrumentation data collection. That’s where our boundaries are specifically set. 

    We will never do anything about visualization. We’re trying to make you gather your data and then after, you can choose how you want to interact with that. There’s a little bit of that boundary that I think are being set around projects of where things go, because you can either try and almost be just kind of gobbling everything up, everything around you. Or you set your boundaries and then you have to let something else track there.

    Alan Shimel: Sorry, I think that’s a CNCF decision. They have to make, like for instance, how many service match projects do you are under CNCF.

    Constance C.: Well, so that’s a no kingmakers rule. Like we do have an explicit rule about there being no kingmakers, because we do want to offer things right. Because I’m biased that I love Envoy, but for some people Linkerd works great. And that’s amazing because there’s all these things there. Some people really love Prometheus, I think open source is great. 

    Metrics is not necessarily equal there. There should be different option because things work better for other there. But it doesn’t mean that we can’t maybe say have like, I don’t know, a few CNCF Stacks. Like these 10 things work great here, these 10 things work there. And this is actually going to be more on the engineers, because CNCF does have more of the model of the open governance so you choose your own adventure. 

    But then you see a little more of, it’s this great thing about open source, you choose your own adventure but we also need to think a little bit more like PMS, because marketers and salespeople are thinking about what the future will be. Because if we just implement the next thing, like go faster build all those things there, you’re not going to have sustainability to project.

    Because I think the thing that we’re taught, everyone’s like, okay. This is almost sold as this is the last big project you’ll ever adopt. 

    Alan Shimel: Agree.

    Constance C.: But we’re not. And if we don’t think about the future, sorry, everyone, you’re going to have another rewrite in 5 to 10 years.

    Alan Shimel: Guys, we’re about out of time, I got to ring the bell here as I said in the beginning of our talk. But you know what, look. I didn’t take the prerogative of taking the last word because I’m the host. We are in a golden age of open source innovation right now. And we have been for the last couple years. And I think CNCF in similar kind of foundations have ushered this golden aged. 

    And there’s a lot of companies including Logz and Splunk, and many, many others who are constantly innovating and pushing what we can do off of this foundation, and I mean foundation of open source. And I don’t know how long it’ll last until we go into a post-cloud native world, I don’t know. I may not be here, but I do think when we look back on this, it is a golden age.

    And it’s going to be very interesting to see what the next short-term, because who knows long-term, what the next short-term future holds and especially for observability. Observability is kind of the poster child right now for what we can do. That’s my poster post right there. Anyway, on that note, I think we’re going to close this one out. 

    Constance, thank you so, so, so much for joining with us today, this was great. I apologize, I had that little network outage there and I don’t have you guys up on my usual monitor. So I keep looking over here to my computer. But thank you for joining us. Dotan, as always, a pleasure to have you on my friend. I look forward to being able to see you soon. Yep. And then of course, Mitchell. You want to take the last word, Mitchell, and we’ll wrap it up.

    Mitchell Ashley: I’m going to leave it right where you said it. You just said it perfectly. Let’s end there.

    Alan Shimel: Okay, then. Hey, we’ll be back in just a couple of weeks, two weeks I think, with our next Observability At Scale. It’s a great series if you’re interested in observability. Check it out. But for now, this is Alan Shimel, TechStrong Group. Have a great day.

    [End of Audio]

     

  • NextArch Foundation Launched to Focus on Microservices

    NextArch Foundation Launched to Focus on Microservices

    The Linux Foundation this week launched the NextArch Foundation to define frameworks and specifications that promote interoperability between microservices that span multiple cloud computing environments.

    More than 40 organizations, including Tencent, Arm, Kong and Ampere, have committed to contributing to 10 technology projects in 2022 in addition to frameworks for three vertical industries.

    Mark Shan, chairman of Tencent Open Source Alliance, said within the next five years the NextArch Foundation expects to have defined a set of standards and best practices that will have been widely adopted.

    The NextArch Foundation’s mission is to provide a home for a set of emerging technologies that would otherwise not get the attention they deserve within the confines of an existing consortium, said Shan.

    NextArch Foundation’s areas of focus will include how continuous integration/continuous delivery (CI/CD) platforms are employed to build microservices and reducing latency between microservices residing in highly distributed computing environments, added Shan.

    It’s not clear where the efforts of NextArch will begin and end as compared to the Cloud Native Computing Foundation (CNCF), another arm of the Linux Foundation that focuses on platforms such as Kubernetes that are widely used to build and deploy microservices-based applications based on containers.

    In general, however, it’s already apparent that microservices are at the core of most digital business transformation initiatives that span everything from artificial intelligence (AI) projects to edge computing environments. The global digital transformation market size was valued at $336.14 billion in 2020 and is expected to grow at a compound annual growth rate (CAGR) of 23.6% from 2021 to 2028, according to market research firm Grand View Research.

    As organizations embrace microservices to build and deploy software, the need to establish a set of DevOps best practices to manage them given the level of dependency that exists between multiple microservices within any application.

    Challenges associated with managing microservices are not likely to deter organizations from adopting them, but they do slow down the transition. Most new applications are intended to run on mobile devices that access cloud services, and typically run more efficiently when constructed as a set of microservices that scale up and down based on usage. The levels of agility and resilience that microservices have are crucial to any digital business transformation initiative, so the general expectation is that the percentage of microservices-based applications that will be running in production environments will increase sharply.

    The challenge, of course, is that, over time, microservices-based applications are more challenging for developers to build and maintain than a traditional monolithic application, though they are easier to update. It’s not likely microservices-based applications will replace existing monolithic applications overnight, but many existing monolithic applications are being converted into a set of microservices that makes it easier for developers to reuse the core functionality of that application.

    Of course, it’s still early days as far as microservices in the enterprise are concerned, but as developers shift toward a new way to build software, the need for a wider range of de facto standards should become more apparent.

  • 5 Tips for a Successful DevOps Implementation

    5 Tips for a Successful DevOps Implementation

    More businesses are turning to DevOps implementations to deliver software and updates with more efficiency. As much as business leaders know DevOps can help them to be faster and more agile, many of them are unsure how to get started. Many businesses even begin implementing DevOps only to run into unforeseen problems that hinder the process. 

    While DevOps can be beneficial, you do not want to make changes lightly. You need to know what you are getting into and prepare for ways the change can impact your company. Having a good team and providing them with the right infrastructure-as-code (IaC) tools will help, but there is so much more you need to consider before starting. 

    Here are five vital tips for successful DevOps implementations.

    Evaluate Your Processes

    You need to start by learning what DevOps is and what it means to your organization. Everyone on the team needs to understand what DevOps can do and the types of problems it can solve for your company. 

    Evaluate your entire development cycle from beginning to end. Look at the different processes that go into development to find different bottlenecks and inefficiencies. By finding the areas where your current processes come up short, you will be able to identify the areas to concentrate on when implementing DevOps.

    Collaboration and Communication

    Collaboration and communication are at the heart of successful DevOps implementation. You can get all of the best DevOps tools available, but if you do not have teams ready to work together and communicate, you will not achieve the full benefits of DevOps implementation.

    If you are going to succeed, you need to instill a strong sense of collaboration and cooperation among teams. Beyond that, you also need to provide your team members with tools that will help them communicate and collaborate. Communications platforms and collaboration software can be just as important as the tools that contribute directly to development.

    Set Goals

    DevOps implementation is not just about getting the right tools and changing processes. You need to assess the operations of your company and set goals for the implementation. If you are just buying new tools without setting a new direction, the implementation will not result in that much change. 

    Another good tip is to start with one smaller, more easily achievable goal and then scale up from there. Changing everything at once might be too much of a disruption. Instead, select a smaller goal and once that is achieved, move on to goals that are larger and more complex.

    Automation

    Automation is a big part of what makes DevOps beneficial to modern businesses. If you are going to achieve the full benefits of the change, you should automate as much as you can. Whether it is scanning code for misconfigurations or automated testing, you can do a lot to make your operations more efficient by using a variety of different automation tools. 

    You may also want to take automation one step at a time. If you try to dive headfirst into automation, it might be a bit too much for your team to keep up with. Instead, take a manual process that has been taking a lot of time and toil from your team and implement an automation tool. Let your team monitor for a few weeks to see how it goes and then take what you learned and apply that feedback to automating more processes.

    Know Your Key Metrics

    Set your key metrics from the very beginning of the implementation process. Without metrics, you won’t be able to keep track of the progress you make. A lack of tracking will also make it difficult to identify issues that might be holding back your progress. 

    Most organizations will need to focus on three points: Lead time, deployment time and mean time-to-recover. Beyond that, try to think about your users when defining key metrics. What do they like? What do they care about? What makes for a more pleasing experience?

    DevOps implementation is not simple. By taking the right steps to prepare, you can avoid the worst problems and put yourself in a position to respond well to the issues that do come up during implementation.

  • DevSecOps Tech Radar Highlights Diverse Tooling Adoption

    DevSecOps Tech Radar Highlights Diverse Tooling Adoption

    The DevSecOps Technology Radar showcases the opinions cloud-native groups have about DevSecOps tools. To review, The Technology Radar is a periodic report from the Cloud Native Computing Foundation (CNCF), a burgeoning host for new open source projects that are becoming pivotal underpinnings to cloud-native architecture at large.

    Just like previous radars, the DevSecOps Technology Radar was assembled using feedback from the CNCF end-user community, which is a group of over 155 organizations that use cloud-native technologies to build and distribute their services, spanning startups and enterprises. The Radar report organizes popular tools into adopt, trial and/or assess status, based on a technology’s active use and its maturity.

    ‘Adopt’ means these tools are stable and come highly recommended for use in production systems.

    ‘Trial’ means users have had some success with these tools, and it is recommended that others consider them.

    ‘Assess’ means these are promising tools that are good for solving specific needs.

    So, why was DevSecOps selected as this Radar’s theme? According to the Radar team, DevSecOps was chosen because it is highly relevant to current development focuses. “We realized that more than half of the tools we were using in CI/CD were actually security-oriented,” said Sergiu Petean, head of DevOps at Allianz Direct. This goes beyond DevOps—”this is definitely an uppercase ‘Sec’,” he said.

    “More than half of our time is spent discussing designs around security,” said Keith Nielsen, director, cloud architecture at Discover Financial Services. Financial services especially must find innovative methods to balance speed-to-market with security. Nielsen also cited the rapid rate of change in this space, which requires companies to continually reevaluate their security posture.

    DevSecOps Radar Results

    So, where did the CNCF end-user community place popular DevSecOps tools? The Radar ended up rating 16 tools across the three different levels:

    • Adopt: Istio, Sonarqube, Artifactory, Hashicorp Vault, Calico/Tigera, Terraform, ArgoCD and OPA.
    • Trial: XRay.
    • Assess: Cilium, Harness, SonaType Nexus, Hashicorp Sentinel, GitHub Actions, Linkerd and Trivy.

    The above results were based on 252 votes from end users across 35 tools, with most respondents coming from large companies across multiple sectors.

    Out of these tools, Terraform collected the highest number of ‘Adopt’ votes. This technology aids the move toward infrastructure-as-code (IaC) with declarative configuration files, which is important for DevSecOps automation. Tied for second place were Hashicorp Vault and Artifactory, indicating that secrets management, access control and continuous updates for security are pressing requirements.

    Other open source tools on the Radar are providing impressive ways to unify security controls. For example, OPA is a universal layer for applying policies that could help unite fragmented authorization across applications. Or, service mesh options like Linkerd and Istio place a sidecar proxy around applications to centrally manage access control, observability and networking across a large microservices ecosystem. Such projects are programmable to fit into the DevSecOps pipeline.

    Analysis

    In general, the radar team was somewhat surprised to see such a high number of security tools in use today. Some tools compete in the same arena, making hunting for the right tool challenging. “The sheer number of products is a bit staggering … what we’re seeing is a highly fractured strategy from different companies,” explained Nielsen, who said he expected more consolidation on capabilities.

    After evaluating the DevSecOps tooling landscape, The Radar team organized their core findings into three main takeaways:

    Usability is compromised at the expense of security. Security features are necessary, but they often come at the cost of usability. The DevSecOps Radar team found that the developer experience for most tools is poor. Many disparate tools must be stitched together to have a truly automated pipeline which is cumbersome to implement. “The tools are getting better, but still there’s not a cohesive, prescriptive way to develop for the cloud,” said Nielsen.

    The pace of change in the security space is rapid. Major cloud providers continue to rapidly release new services, which has led to an explosion in new tools. Simultaneously, the speed of innovation is accelerating within companies, opening up unique security needs. Combined, this means the security landscape is constantly shifting. “Security tools are trying to keep up with the services and how you host them and run them,” Nielsen said. “We definitely weren’t expecting to see so many newcomers in the security space,” Petean agreed. Though DevSecOps is spoiled for choice these days, “it’s always better to have options,” he added.

    Microsegmentation is important but presents a significant challenge. Many great tools enable companies to improve how they segment traffic, such as service mesh tools like Linkerd and Istio. However, implementing cloud-native microsegmentation is bumping against legacy implementations for established companies. Between API gateways, service mesh, edge firewalls and Kubernetes federated firewall functions, DevSecOps has much complexity to deal with, Nielsen said.

    The DevSecOps Technology Radar: In Review

    In recent years, cloud-native technology has placed increasing emphasis on solving the security puzzle. Companies continue to reevaluate their security approach in light of cloud-native strategies, novel vulnerabilities and new compliance requirements. DevSecOps is proving to be a useful, if not necessary, practice to maintain both security and rapid software delivery.

    With new security tools and unique approaches coming out so frequently, it’s easy to have “analysis paralysis.” However, DevSecOps must start somewhere. And, for those who trust the opinions of the CNCF Technology Radar, you now have some helpful insights on where to begin.