Category: DevOps and Open Technologies

Editorial Calendar for February 2016

  • EP 3: Open Source and DevOps

    EP 3: Open Source and DevOps

    In the third episode of our DevOps Unbound streaming broadcast on TechStrong TV and devopsy.kinsta.cloud’s sister site Digital Anarchist, Alan Shimel is joined by Kevin Dunne of Tricentis, Tracy Miranda of CDF and Alon Girmonsky of UP9 to answer the question: Can we have DevOps without open source?

    The video is immediately below, followed by the transcript of the conversation. Enjoy!

    Transcript

    Alan Shimel: Hi everyone. Welcome to another episode of DevOps Unbound. DevOps Unbound is sponsored by Tricentis, so many thanks to them. But it’s our show here at MediaOps and we try to explore, you know, just some of the areas around DevOps and not just dev and ops but DevOps as a framework for the way things are being done, the way application and software is being made and deployed today, the way it’s being hosted, the way it’s being operated and maintained and observed, and so many other aspects of how we’re doing technology. 

    It airs every other week and this episode is episode number three of DevOps Unbound. Hope you’ve enjoyed the first two.

    With that let me introduce to our panel today. First of all, well, it may not be right or left on your screen, because so it makes no sense to tell you how it is on my screen, but let me first start off with my friend, Tracy Miranda. Tracy is the executive director of the Continuous Delivery Foundation, which is a Linux Foundation and a sister organization to Cloud Native Computing Foundation. Got that all out of the way. Tracy, welcome. 

    Tracy Miranda: Hi everybody and thanks for having me Alan. 

    Shimel: My pleasure to have you on Tracy.

    Joining Tracy and I today, I have Kevin Dunne, who has been on our TechStrong TV in the past. Kevin is with Tricentis and runs their open-source communities, as well as some of the open-source projects that are now under their management, some by acquisition over the years, some they’ve developed themselves. But Kevin, welcome to DevOps Unbound. 

    Kevin Dunne: Thank you, Alan. Happy to be on and glad you’re having us back with you again. 

    Shimel: That’s our pleasure. 

    Then last by not least is my friend, Alon Girmonsky. Alon is the CEO/founder of UP9, that’s U-P-9. He’s going to tell us about that when I ask everyone to introduce themselves here in a moment, but Alon was also the founder/CEO of BlazeMeter, a very well known open source or player in the open-source testing world acquired by CA, which of course became Broadcom. 

    So, today’s episode ladies and gentlemen is around DevOps, cloud native and open source more importantly, most importantly, and how important open source is to this whole DevOps movement and this whole — Someone today, I had someone from Deloitte Cloud on and I was doing an interview called it a “revolution” that we’re seeing, especially since COVID, right? IT has undergone a revolution of cloud transformation, digital transformation, acceleration and so much of it is made possible by open source. So that’s our subject today. 

    Before we hop into it I wanted to give each of you like 30 seconds, just in case our audience wanted more than the cursory kind of introductions I did. Why don’t we start with you Kevin? Give them who you are, your background and we’ll jump through all three of us from there. 

    Dunne: Yeah, that’s great. My name is Kevin Dunne. I’m with Tricentis as Alan mentioned. My main focus is on our open source and sort of community products. I’ve been with Tricentis for seven years. Prior to Tricentis I was at QASymphony, which merged with Tricentis about two years ago. So very passionate about open source, it’s been sort of a new area that Tricentis has been getting into. 

    There was a lot of work around Flood, which was built on top of JMeter and Gatling and now Tricentis has acquired TestProject which is built on top of Selenium and Appium and most recently SpecFlow which is an open source project based on cucumbers. 

    So I’m really excited to see how these tools are helping to kind of shape the Tricentis roadmap and sort of broaden the community in the open source supports, so very exciting projects going on so far. 

    Shimel: Alon, how about you?

    Alon Girmonsky: Hey everyone and Alan thanks for inviting me. I’m a big fan both of Tricentis and obviously the CD Foundation. Alon Girmonsky as you mentioned my current company UP9, which I have the privilege of being cofounder in, deals with microservices and cloud native in respect to testing. Testing is something that is becoming harder and harder to do, especially when you know you talk about service testing and API testing. So UP9 provides a solution for somewhat autonomous, mostly automatic you know service testing. 

    As you mentioned prior to that my previous company, BlazeMeter, was all about load testing and stress based on the open-source tool JMeter. You know I had the pleasure over the past 10 years to be involved in the open-source community and understand or at least try to understand what’s the place of open source and why is it so important specifically for developers. I’m very happy to have the opportunity to discuss this in this forum. 

    Shimel: Last, but not least Tracy how about you?

    Miranda: Tracy, I’m the executive director of the Continuous Delivery Foundation or CDF, which is an open-source foundation with the mission to improve the world’s capacity to deliver software with security and speed. The CDF is home to the open-source project Jenkins, JenkinsX, Tekton and Spinnaker. 

    Before that I was leading open-source community at CloudBees. In general I’m a 15-year veteran of open source, so I love all things open source, open innovation and building communities. 

    Shimel: Excellent. So folks, here’s my opening premise and we’ll let — it will be our jumping off point. 

    I also have a long involvement in the open source world. I first got involved in open-source security related projects 2002, 2001 and we were blessed in security to have some amazing open-source tools at our fingertips. Products like Snort, which was developed by Marty Roesch, who then started Sourcefire, acquired by Cisco, was the de facto standard for intrusion detection. The NMap and Nessus scanners, Renaud Deraison, who’s cofounder over at Tenable Network Security and Ron Gula, who started Tenable with Nessus. NMap has been out there. There were many others, ClamAV and so many commercial companies actually sprang up just by using that open-source software under the covers putting a UI on it and stuff. 

    So I’ve seen open source. I remember talking to CIOs and I remember speaking to you know a DOD agencies about their use of open source and they would tell me that they don’t have any open source, they don’t allow open source and then you would go look at what their developers were working on and it was all open source, you know it was Linux and everything else that they had, just didn’t bother telling them about. So I’ve seen open source become from a gorilla-type of operation in many environments, to the dominant form of software and software development today. 

    When I got into DevOps it became really apparent to me and this is my jumping-off point, we wouldn’t have DevOps without open source. I just don’t think — I think so much of what we can consider not only the tools of DevOps, so specific open-source projects, but the culture of DevOps, the collaboration, the sharing of knowledge of DevOps is based upon that open-source mindset, on the open-source culture that you couldn’t have DevOps without open source. 

    Tracy, you ran the open-source community of CloudBees first and only ED at CDF, part of Linux, what’s your feeling on that? Why don’t you kick us off? 

    Miranda: Yeah so definitely as a big fan of open source I’ve seen how it’s been really instrumental to DevOps. I think you know one of the things that open source does really well, like sometimes people focus just on the code, but it is really all about that collaboration, how people work together. I think at the heart of DevOps really what it’s trying to get at is building bridges, getting people to work together even if they are in slightly different spaces, whether you know it’s DevOps COPs who are trying to bring security practitioners closer to developers or ops people closer to developers. I think open source really captures that spirit of collaboration and that’s why it’s become almost synonymous with DevOps. 

    Shimel: Kevin, Alon, what do you think? 

    Dunne: Yeah, I mean I would definitely agree with that. I would say one other thing that you know of the many benefits of open source, right, is kind of just the transparency and visibility. I think you know a lot of companies were finding like DevOps in the CI/COVID process especially is kind of integral to business success, right? So a lot of companies are very hesitant to put a tool in place that is going to be that critical to their success unless they’re able to you know fully understand what it does, you know any limitations that it has, you know extended where needed and you know kind of have that ability to build on top of it as well.

    I think way too many companies kind of got burnt a bit by you know solutions that were completely closed source, didn’t allow customization and things like that. Then you know they kind of get stuck when you know something breaks. So they reach the limits and the vendor is not able to you know update that for them. So I think that was one of the big things about you know the whole Jenkins’ community, right, as you just see how many plug-ins and other you know ways people come up with of extending the tool that you just would never see from a closed-source product, right. So I think that’s been something as well as just kind of like the ability to have the transparency into how something works and be able to extend that to fit your own needs is pretty critical to DevOps success I think. 

    Shimel: Yeah, I, I agree. Alon, you more than I think even Tracy and Kevin I mean you built a business with a successful exit on an open-source kind of business model if you will. You know what’s your view on… Is, can, is it a chicken and egg? Can you have DevOps without open source?

    Girmonsky: I think if we go back even in more than a decade there was open source and there were commercial offerings, right? There was an inherent conflict between the two. 

    I think that if you fast forward for today it’s about product market fit. I think open source needs to be part of any product as long as it involves developers, maybe even more than developers, but developers as Kevin kind of mentioned we talked about transparency I think developers need to have control. They cannot rely on someone else to have a black box. So anything from transparency, having control, not having dependency. You know if you think about speed to market the more control you have your ability to change quickly you know the faster you can go.

    So what we’ve learned over you know the past decade the developers they need to have the open source not because it’s free. I think that’s the last aspect of open source. It was in the past it was free and a poor man’s solution. Today you want to have open source because you want to maintain control, visibility, transparency and reduce the dependency on other vendors. That’s kind of my key learnings you know from my experience. 

    Shimel: Yeah. I mean that reminds me of something I’ve always been taught and heard about open source and that is look with open source it’s free as in freedom and free as in beer, right? It doesn’t cost, it’s free as in beer, right, free beer for everyone. But really though that was a big driver, you’re right Alon that was a huge driver and remains a huge driver that people can just pick up this software and run with it. And if at some point they want to buy an enterprise license or an enterprise version or go to a vender that has a supported version they can pay for it, but they can use it for free today. But more importantly than that is the freedom as in freedom, right? In that it gives them the transparency, it gives them the ability to see exactly what it is they’re getting themselves into as Kevin pointed out. 

    But to me that’s almost turned the original problem on its face. Tracy you’ve been in these, in this open-source world longer, as long enough, right? In that originally that was also one of the big shortcomings of open source, right, is that everybody, everybody sees the source code, hackers or whoever will more easily exploit that source code or can break in or can see what’s going on. And whom am I going to call, Ghostbusters, right, for support and to fix this thing, right? 

    And so how can I as an enterprise you know based my business on open source? Is it reliable enough, secure enough, supportable enough? But yet we seem to have overcome that. One of the reasons I think, Tracy, is because of the advent of foundations like the CDF, right? So talk about that if you will a little bit. 

    Miranda: Yes, yeah, no absolutely. Like I think well first of all I want to make it clear like you can still have DevOps with a proprietary CI/CD stack and there’s plenty of offerings out there. But I think the key thing where open source plays a big role is it’s really in driving the innovation across the industry, making it so people can leverage it in different ways, combine it and get that interoperability to you know produce things that solve their problem. 

    And the way foundations have really helped is dealing with some of the issues or you know sometimes there have been misconceptions with open source, but sometimes they are genuine problems, which you know fall in the gaps of well maybe the developers on the projects want to focus on the code and there are other things we can do across multiple projects to help you know whether it’s focusing on security or focusing on building communities that are diverse. So open-source foundations have just added that extra layer to give the industry confidence that these are trusted projects that are being well sustained and people can rely on them to either build products or use in their own CI/CD stacks and really just kind of push, push things further I a good, positive direction for the whole industry. 

    Shimel: I’m going to comment or not disagree, but I wanted to distinguish and clarify something you said there Tracy and that is yes there are closed source or commercial or whatever you want to call it CI/CD solutions out there. But even they and we report on them at devopsy.kinsta.cloud right, even they rely on open-source integrations, open-source components. 

    I mean you look at Alon with what you’re doing with UP9, with UP9, with what you did in BlazeMeter. Kevin when he talked about the three or four different open-source kind of based tools that Tricentis is doing, they’re all based on open source. When you go under the covers they’re closed source but they’re based on open source, things like Selenium and these kinds of things. 

    So you know even in the CI/CD space and I’m not here to name names, but we know the closed source you know CI/CD players. I don’t think they’re product works well if you didn’t have an open source kind of framework that they plug into. I think it’s especially true in the testing world. I think Alon you’re living proof of that, right? Could you have had a BlazeMeter without having some of those great open source? Kevin all of the projects you mentioned are based on open source, right? Can you have any of them without open source? It’s a real you know … 

    And guys don’t wait for me to ask your name, feel free to jump in here. 

    Dunne: I think it’s a good point. It’s becoming more of the model like Alon was you know mentioning is it’s kind of you stand on the back of giants a lot of times, right? So I think you know you see a lot of the at least in the testing space, right, BlazeMeter, Sauce Labs, BrowswerStack, you know a lot of these bigger companies they’re succeeding by basically doing what you were talking about with the security tools 10, 20 years ago, right? 

    Take an open source community and kind of you know build something on top of that, right, that provides value, but still is kind of using this known currency of this open-source project. I think people like that because you know by the time they get to buying that commercial solution they’ve got a lot of confidence in the open-source components that have reached a you know critical kind of juncture where they’re bought into those tools enough that they’re willing to invest in something you know commercial to help support it and you’re solving real problems, right. 

    You mentioned you know free and open source being free, but you know everyone knows one of the big challenges with open source is just the amount of time that you have to invest a lot of times to customizing it, deploying it, you know patching it, upgrading it, dealing with security concerns. And so you know a lot of these vendors come in and they solve a real problem, but I think it is a little bit of a you can put the cart before the horse, right? 

    The people need to adopt the open-source tool, see that it works, feel the pain of you know what it’s like to manage it on their own and then you know usually graduate to these commercial solutions. I think when people come and try to present the commercial solution upfront without that open-source community you know it kind of fails because you haven’t proven the value yet, right? And it’s a big jump for people to make and that’s you know been a good thing in the market having open source. It’s kind of put the power in the hands of the users that you know users get to call the shots first and the user expects you’re going to give me some value in this open-source tool before you start to charge me an arm and a leg, right? 

    Miranda: I like the analogy, free as in free puppy, so we can have it, but you still have to take care of it, feed it, and keep it from pooping on the carpet. 

    Dunne: Exactly yeah. 

    Girmonsky: Yeah. 

    Dunne: And that takes a lot. 

    Shimel: Alon? 

    Girmonsky: I think I’ve learned a good concept a couple of years ago, kind of the open core concept. It wasn’t there ten years ago, but then you understand okay this is what, what we’ve been doing all along. The ability to open your core, the core of your platform, the core of your offerings. So you know your user will have more flexibility. Maybe the use case is a bit different than what you had in mind initially, right, by having an open core or an open-source project that you’re basing on, right, you allow the users to build their own use cases and then bring value that again the product market fit is in my view better than anything that has close and this is what you can do with it and that’s it. 

    Shimel: Agreed. So Tracy you know the CDF is a great model for the kinds of stuff we’re all three of us are talking about, right? You have in essence this four main properties or projects that CDF manages. Each one of them is closely tied to one or more commercial entities that solutions are based upon, right? But yet the community revolves around the CDF. By the way it’s the same thing in the CNCF, right, where you have Kubernetes and you have you know a lot of Kubernetes service providers that manage Kubernetes, but the community lives around CDF. 

    What are you seeing here? Like where is that handoff? Where is the you know passing the baton from that open-source user to the commercial entity? 

    Miranda: Yeah, no I think that’s a great question. What we see, well certainly what I’ve seen over the last 15 years is it’s an ever-changing line if you like. So when we talk about innovation you know we have these open-source projects and we don’t want people to be reinventing the wheel over and over again. Like take something like pipeline orchestration. You know when it gets to a point where that’s pretty ubiquitous then that’s a good thing to have as an open source you know kind of standardized in one project. 

    Then we can move up the stack into new innovations and vendors can lead that charge of take open-source projects and try to build things on top of that that add new things to the industry. As those become more and more ubiquitous then you know it probably eventually will get an open-source solution and then people will sort of move onto the next thing. 

    So I think it’s always kind of this tide as such where the more kind of proprietary things will test out the beating edge and where they can get value for using some things that haven’t been done and then open-source projects become like the infrastructure, the roads, and the bridges and things like that. 

    Shimel: Fair, guys any comment on that?

    Dunne: Yeah I think that’s true. It’s kind of tough though puts vendors in a tough position, right, where it’s like you know you have to – it really raises the bar, right on delivering innovation, right, because you know all of the… It’s kind of a commoditization, right, that you know things that have been in the market for 10, 20 years you know the market, the commercial market for those sort of erodes and you know it gets replaced by open-source tools. And so it’s always kind of playing that game of being the you know next hand-up on the baseball bat. 

    You know it’s just like your always – if you stop innovating there’s… I think that’s why you see you know the SAS model being really popular with a lot of these software companies is just they need to keep innovating, right, and you can’t just like you know expect to sell a tool, a licensed tool and you know have a customer use that for ten years and collect maintenance from them. Like these are – all the tools that are you know demanding good budget from companies are really you know evolving quite quickly. So it’s really put a lot of pressure on vendors to innovate, otherwise they’re going to become stale and replaced by open source pretty quickly, which is a good thing for the uses of these tools I think at the end of the day. 

    Shimel: I also think that’s to Alon’s point that’s why OpenCore is such a great model for this, right?

    Dunne: Yep. 

    Shimel: Because you can’t – you can keep innovating at your core and then have your freemium, premium, whatever you want to call feature set kind of revolving around that core and keeping people, “Okay now that’s why I should pay money.” 

    Dunne: Right. 

    Shimel: So –

    Miranda: And I think –

    Shimel: Go Tracy. 

    Miranda: And I think we’ll increasingly see companies kind of use that to change the dynamic in the market and then they can use open source almost as a weapon to drive commoditization by saying, “Okay well this thing that was a premium feature for a number of vendors, we’re going to open source our vision” and you know suddenly change the dynamics. So it is real interesting because, yes, I think SAS services will make a big difference and everybody in the industry has to be on their toes constantly. 

    Girmonsky: Absolutely. By the way a way to compliment this is think of you know we used to live in an industry where the big vendors provide point tools, value, do something that you haven’t done before. Today, with developers it’s very hard to think of having an infrastructure that is not open source, right? So let’s assume the entire infrastructure is open source whether by vendors, by developers, or by big cloud like you know Netflix, Amazon and others. Then the next layer is efficiency and productivity. Then the commercial vendors they can provide impact by, you know, this is what I have done in BlazeMeter. It wasn’t — BlazeMeter wasn’t a load-testing tool. I think Flood was the same. It’s JMeter was the testing tool. 

    Dunne: Yep. 

    Girmonsky: BlazeMeter back then was all about making JMeter more efficient and the developers more productive. So suddenly LoadRunner wasn’t like, okay there is an alternative to what was de facto commercial tool, okay it’s the open-source tool, but as you said open  source on its own it’s pretty tough. So you know here come the vendors that make our lives more productive and use of open source more efficiently. 

    Shimel: Agreed. Kevin, Alon I’m going to put guys on a hot seat here for a second. You know what CNCF has been wildly successful in so many of the projects that they’ve incubated and now graduated and have been tremendous successes. Tracy, CDF is only what about two years old now? 

    Miranda: Yeah, not even. 

    Shimel: Yep, they’ve got four pretty amazing CD tools that are kind of de factor standards, right, that people are gravitating around. With such a strong heritage of open source and testing and continuous testing, automation, why haven’t we seen sort of the equivalent of the CDF in the testing world? Right? Why hasn’t the Linux Foundation or some other, Eclipse or Cloud Foundry or something or a new one sprung up that says, “Hey, we’re going to take these individual projects you know out of hands of individual company and do a community thing”? It may be a case of what, don’t break what’s, don’t fix what’s not broken, right, because it seems to be working the way it is. But why don’t we see a foundation for open-source testing? 

    Dunne: That’s a good question. I mean I think you know if I had to guess the main reason probably is just that you know testers are sort of developers, but sort of testers, right, and like an automated tester kind of falls in between the gap of maybe both and some view themselves as developers, some view themselves as testers. 

    I feel like it’s starting to happen. If you look at some of these tools like Cypress for example. You know the newer tools that are emerging definitely they’re communities have much more of like a developer feel around them and there’s much more structures of the open source contributions and whatnot. 

    But you know historically it was definitely very fragmented and you know the load testers all kind of rallied around JMeter and Gatling and you know the web testers around Selenium and the mobile testers around Appium. There’s really no common thread. I guess it’s kind of more like more siloed, right, versus my guess would be you know these tools like Kubernetes and you know Spinnaker or Jenkins, I mean those are ubiquitous, right? I mean everyone whether they’re building you know a web app you know in Java or in you know C# or whatever can use these tools versus I think a lot of the testing tools because they’re kind of like a thought you know into the decision that’s made after you make some decisions oftentimes on your kind of app infrastructure. That’s why they’re sort of thought of as maybe – they were historically thought of as more of an afterthought and the communities got more fragmented as a result of that. 

    But I think that is starting to change. I mean we know and we talked about these results on the last time I was on with you, Alan, you know that testers are becoming more and more like developers and becoming more technical, right? So I think the tools you know are going to demand to change this community structure, but it will take time but I think we’ll start to see that over the next few years my guess. 

    Shimel: Alon?

    Girmonsky: Yeah I think that if you look at the industry, the open-source industry for past second decade you know prior to CNCF there as Apache, right. 

    Shimel: Yeah.

    Girmonsky: Yeah and it was kind of you know in many ways the collection of open-source tools that if you’re in Apache open-source tool you’re probably in good. And now we see it’s not about open sources, it’s about what you do. Right? CNCF started mostly as Kubernetes and you know other tool and now the CD Foundation I think the focus is kind of all you know on the development lifecycle, as opposed to point tools and use cases it’s about the entire flow. 

    Actually I feel very comfortable. You know if testing finds its right position under CD because in CD you build, you test, right, these are the operations you do day-in-day-out. I would hesitate from having too many because it will kind of defocus many people, “I may do this,” “I may do that,” “Is it different?” Having like you know a family to belong to which is the continuous delivery continues deployment maybe in the future. I think it’s actually you know not a bad direction to take. 

    Miranda: Yes, absolutely and I think you’ve hit spot-on like CD continues deliveries is a process and I think that process does include testing and although we started off with these tools, which are all pipeline orchestration I’d love to you know CDF to future more open-source testing tools and certainly a lot of the conversations we have around interoperability testing comes up a lot. You know people are like, “Why isn’t there a standardized way to show pipeline results, the test runs” and why don’t we strive for that? That’s you know a problem that lots of end users highlight. So yeah, I’m all for you know having CDF be a place where we can have more of the testing community come together, seed, expand beyond kind of the traditional developer or and just make folks feel included in that community. So yeah, we should definitely talk. I like the direction [crosstalk]. 

    Shimel: So you heard, you heard it here first on DevOps Unbound. Alon, Kevin you have her contact info. 

    Dunne: Oh yeah. 

    Shimel: Let’s see some testing projects being part of CDF, right? Let’s bring it together. 

    Alon, the next line of questioning starts with you, but Kevin and Tracy feel free to jump in. So Alon you’ve been there, done that, you did BlazeMeter as you mentioned. You build it on JMeter and some other open-source stuff, but you didn’t want to duplicate what JMeter did, you were supplementing it and building on top of it. 

    Your present company, UP9 is new, not a lot of people are as obviously familiar with it as BlazeMeter. But I remember interviewing you when we first started talking about UP9 and you said, “Look this is for now, right, this isn’t what BlazeMeter…you know we took a lot of lessons that we learned and a lot of things that we saw, but this is for today.” What makes it for today, especially from an open-source perspective, right, that was different than when let’s say BlazeMeter you know first started with JMeter. 

    Girmonsky: Yeah. So BlazeMeter dealt with a problem that was irrelevant for 2010 or 2012, it was again it was kind of moving to the cloud, the Web-scale system, but mostly around service oriented architecture, right, and WebUp monolith of the time. At this time let’s say 2020 and going forward we’re talking about cloud native and containerized system. The industry took an approach to build the system that scales horizontally, right, with microservices. When you have than a few microservices you bring in Kubernetes or Docker Swarm or ECS sometimes, right, to orchestrate. 

    But this entire new tech stack it is far more complex than it was n the past and still you need to test. You have you know you call it not reliability, you have to have reliability case. You know you develop software you don’t want to develop faults that become failures, right? So and again it’s far more scalable in the past. If we used to do one service, now we have 200 services, dependencies, individual roadmaps, what do you do? 

    So UP9 takes care of it. UP9 provides a way to first test each and all microservices, like to provide the complete test coverage, but also do it you know use machine learning to offload a lot of the work from the developers. 

    Now to connect this with open source again I’m a strong believer that it’s not this or that, it has to be both. You have to start with open source because you first need to onboard your users, right? The only way to onboard them for them to have confidence is by giving them open-source tools and not because it’s free or because I want to make a name for myself in the open-source community, because that’s the only way to start you know the operation and then you make their lives, the developers’ lives more efficient.

    So this is UP9, UP9 start with open core solution that enables testing microservices in a Kubernetes ECS Docker kind of environment, but making it no matter how many services you may have, how many endpoints, whether it’s thousands of endpoints and hundreds of services you know you get things going pretty quickly, because you know this is how you would do things these days. 

    Shimel: Understood. Kevin, Tricentis I mean Alon’s example applies directly to Tricentis, right? You have three or four projects that you’ve spoken about, acquisitions that you manage, you know you’re managing these open-source communities and project, but yet there’s, you know there’s commercial or closed-source Tricentis. Where is the integration, where’s the overlap, where’s the leverage from one to the other? 

    Dunne: Yeah it’s a good question. I mean I think we’re kind of starting to see two different I would say maybe like communities of practice you know coming out of large IT organizations, right? You have definitely what Alon was talking about which is like kind of people building these I would call “systems of engagement,” right, which are customer-facing apps, that you know have all of these challenges around how to deal with containerization. So that’s where we really target a lot of our open-source tools, right, and open-source friend tools so that those people who really developing these web apps, mobile apps, you know web services in-house. But we’re also seeing that you know there’s this other large community as well, which is kind of all the systems of record, which should be maybe your main-frame systems, your package apps, like SAP, Salesforce, you know Workday. 

    And some of those communities, like some companies are starting to really start to try to do CI/CD and things like that around Salesforce and some of these more cloud-friendly apps, but for the most part a lot of those communities that kind of work differently they have different expectations, they usually are less technical and a lot of times would actually prefer sometimes a closed-source tool to kind of match the closed-source ecosystem that they’re supporting. 

    So that’s where I think you know when you look at it you kind of wonder, “Well hey how, why does Tricentis have both closed-source and open-source tools?” But I think you’ve got to have the right tool for the job and the right tool for the user, right? And so we see that you know you can-to-go and bring a highly technical tool to you know a nontechnical user that’s trying to test you know a very legacy package application. And vice versa you can’t bring you know a commercial tool a lot of times to a developer that wants to be you know able to write code and you know implement it into their DevOps pipeline and things like that and force them to take that approach. So rather than trying to you know have one or the other group suffer we’re sort of trying to bring together a platform which provides you know different tools for the different approaches. 

    Shimel: Tracy how does that jive with what you see with some of the pipeline tools and projects and companies in the CI/CD world with CDF?

    Miranda: Yep, no I think it’s really interesting and it was great to hear that distinction. So when look at cloud native and particularly Kubernetes and things sitting on top of that, I think the state is today that you still have to be highly technical to stitch a system together and to make it work and to keep up with all the updates. So it’s far, far away from being used very effectively by folks who are less technical or folks who want to be a bit more removed and just get a certain job done, but still take advantage of you know all the benefits of these containerized cloud-native systems. 

    So I think where the open source has to evolve there is to make that a lot easier. We talk a lot about kind of the developer experience and what that looks likes, because today like it’s really confusing if you had to come in new and figure out you know what tool to start with, how to put it together, take Kubernetes itself it’s got hundreds of distributions, which one do you even start? And then there’s a lot of confusing information as well for how you make decisions. 

    I think there’s a lot we can do in open source to drive some clarity on you know what are you trying to achieve, how do you get there, and how do you make the tools work together effectively and not leave it as an exercise to the user to figure out what goes with what. 

    Shimel: Agreed, agreed. Guys were coming … You know I could sit and talk about this with the three of you for the rest of the day, but we’re coming up on 45 minutes, which is kind of our cutoff here. Tracy you’ve heard what Alon and Kevin have said. So first of all guys I’m going to hold it to you. I’d like to see you reach out to Tracy and start and I’ll help, I’ll introduce some of my other friends in the open source and testing world and you have my commitment Tracy on that. I’ll do an e-mail intro. I’d love to see some testing projects become part of CDF. I’d like to see it expand beyond the pipelines to the real tools that folks are using in these things. 

    But you know Tracy CDF as you mentioned is less than two years old. We just last week we had KubeCon CNCF, Virtual Con, whatever I think it was called, what’s in store for CDF? What should our listeners be looking for? 

    Miranda: Yes, yeah, no, we are having our first big cdCon which will be virtual. It’s a first stand-alone event we have for bringing all the communities together. Because of COVID you know we’ve had to pivot quickly to going virtual. So you know it’s not going to be the same as in person, but it is going to be a really good way to bring people together and get that interaction. 

    So cdCon which October 7th and 8th and registrations are open now. That will be yeah just a really good way to come in and figure out what is the state of continuous delivery today and lots of different tracks on specialist topics and cloud native and of course lots and lots of open source. 

    Shimel: Don’t miss it. So that cdCon, October 7th and 8th, highly, highly recommend it. I’m sorry my Apple watch was ringing. There we had to shut that off quick. 

    Thanks Tracy. People can get information at that at CDF? 

    Miranda: Yes, as cd.foundation and find details there.

    Shimel: Great. Kevin, news around Tricentis open-source projects and products. 

    Dunne: Yep, yeah. I think for us you know like Tracy mentioned we’ve got to go virtual this fall as well, but we’re having our annual Accelerate conference this time online. We had one that was focused in kind of May on more of the US area, but we’re having more of a global one to come in October. So we’re going to be showing off a lot of exciting new stuff about a lot of the tools that I mentioned like test projects, SpecFlow, Flood. A lot of new integrations kind of between those tools and the core platforms. So starting to develop that story that I mentioned about sort of one platform to fit various different approaches, whether that’s more of the technical users or the nontechnical users. So that’s definitely one to check out. I’m just checking the date. I think it’s the 21st and 22nd of October is when we’re planning that for it I believe.  

    Shimel: It’s going to be a crazy October of virtual events. Alon, how about you and UP9?

    Girmonsky: Yeah, you know and the UP9 is a new startup. We started almost two years ago, but we are working diligently to complete our software. We already released our first version, but you know the work, still a lot of work ahead of us. So I hope to show you a lot of new cool things down the road. Now we are heads down you know building and building. 

    Shimel: Excellent. Guys I want to thank all three of you for being on DevOps Unbound with us today. I wish we had more time to dive into this. You know there’s so much. Even the idea of having a CDF versus let’s say five or 10 years ago if I told you we were going to have something called a “CDF” you wouldn’t believe me, right? You would think, “What do we need this for?” And you know so we’ve come so far in DevOps and continuous delivery and Kubernetes and — well Kubernetes itself is open source, but this whole revolution honestly around open source and community it’s what makes, it fuels so much of what we are seeing. You know where would we be in COVID without a lot of what we’ve, open source has enabled us to do you know using technology? So I do think it is, it is an integral, it’s a precursor to having the DevOps world that we live in today is open-source software. 

    So, thanks for joining us. We have to have you back on. I’m serious about testing tools in CDF though guys. I think that’s some, long, long overdue. Tracy, thank you and best of luck now that you’ve moved fulltime into the CDF. I know you’ll make great things happen there. Kevin, Alon, best of luck to both of your companies, UP9 and Tricentis. This is Alan Shimel for MediaOps, devopsy.kinsta.cloud. Thank you all. Thank you for watching. We’ll see you on the next DevOps Unbound, soon. Bye-bye everyone.

  • Tasktop Delivers Viz Value Stream Management App

    Tasktop Delivers Viz Value Stream Management App

    Tasktop this week announced the general availability of an application to manage projects based on a flow framework that captures events as they occur across the software development toolchain.

    Company CEO Mik Kersten said Tasktop Viz is the first substantiation of the flow framework in software that provides a value stream management application to enable organizations to more effectively manage software development projects. Tasktop Viz doesn’t necessarily replace project management software as a system of record for billing purposes, for example, said Kersten; however, it does provide a way to visually model the software development environment in a way that makes it easier to identify bottlenecks and potential opportunities to reallocate resources.

    Kersten said from a management perspective, existing approaches to managing software development projects are broken. They depend too much on software developers to manually enter data to update a project management application, which they don’t do as often as they should because it takes time away from developing software, he said. Tasktop Viz takes advantage of application programming interfaces (APIs) to automatically capture metadata from across the software development toolchain, providing instant visibility into software development projects without requiring DevOps teams to engage in an additional coding, Kersten noted.

    Tasktop Viz

    Kersten has been making a case for the adoption of an approach to managing software development projects based on the flow framework for several years. Tasktop Viz now provides IT teams with an application that is designed from the ground up using that model.

    Of course, the concept of value stream management applications has been around for some time. The debate now is not so much over the value of these applications as much as it is which architectural approach to building these applications best reflects modern software development. As the number of dependencies between projects continues to expand, it’s becoming next to impossible for organizations to manage multiple software development projects relying solely on spreadsheets and existing project management applications.

    That issue is further exacerbated by the fact that businesses are now more dependent on software than ever, so any delay to a project could have a dramatic impact on future revenues.

    Kersten said that despite the obvious business benefits, the biggest obstacle to adoption is more psychological than technical. Many software development teams are reluctant to adopt tools that might illustrate how chaotic the software development process within their organization really is. It’s critical that organizations that adopt value stream management applications do so with an eye toward wanting organizations to improve rather than just deploying another tool to measure metrics that are used to increase the amount of code being written.

    Of course, there’s no correlation between the amount of code being written and the quality of the applications being developed. The real challenge is to not just accelerate the rate at which applications are being developed, but also to ensure that once those applications are deployed in a production environment the number of fixes required are kept to the absolute minimum possible.

    — Mike Vizard

  • Survey Sees Open Source Adoption, Quality Gains

    Survey Sees Open Source Adoption, Quality Gains

    A survey of 959 IT leaders published this week by Red Hat finds increased reliance on and adoption of open source software is now being driven by perceived higher quality as much as it is cost concerns.

    The survey finds higher quality of software (33%) was the top reason they chose open source, followed by lower total costs of ownership (30%) and better security (29%).

    Gordon Haff, a technology evangelist for Red Hat, said the survey makes it clear that open source adoption is now being driven from the top-down as well as the bottom-up in most organizations. What’s changed is that open source software is now being used more broadly as more organizations appreciate how open source approaches to building software not only lower costs but also improve overall quality.

    The survey identifies the top three use cases for employing open source software to be IT infrastructure modernization (60%), application development (53%) and DevOps (52%).

    As part of that shift, 56% of respondents said they plan to increase usage of containers over the next 12 months. In fact, on average respondents said cloud-native applications already make up 25% of their portfolio, compared to an average of 36% for cloud-enabled applications and 39% for legacy applications. The combination of cloud-native and cloud-enabled applications (61%) already outnumber legacy applications (39%).

    Over the next two years, 47% of respondents plan to either modernize (17%), rearchitect as cloud-enabled (16%) or rearchitect as cloud-native (14%) their legacy applications. Another 16% said they would sunset or decommission those applications.

    Overall, the survey finds there has been a marked shift away from proprietary software. Only 42% said that more than half of the software they use today is proprietary, down from 55% a year ago. Two years from now only 32% said they expect proprietary software to account for more than half their portfolio. On average, respondents said 36% of their organization’s software is open source, which in two years is expected to increase to 44% in two years. A total of 77% said they would increase usage of open source software over the next 12 months.

    However, barriers to adoption of open source software remain. Security of the code (38%), level of support (37%), compatibility (34%) and lack of internal skills (33%) were all cited as ongoing issues.

    The top three areas in terms of infrastructure where enterprise open source solutions are being used today are security (52%), cloud management tools (51%), database (49%) and big data analytics (47%). A total of 83% said open source has been instrumental in their organization’s ability to take advantage of cloud architectures. Almost two-thirds (63%) said they have a hybrid cloud infrastructure environment today. Among the 37% that don’t, more than half (54%) plan to have a hybrid cloud infrastructure environment within the next 24 months.

    It’s apparent that open source software is now everywhere. The real challenge now is finding a way to manage it all.

    — Mike Vizard

  • Atlassian Extends Road Mapping in Jira

    Atlassian Extends Road Mapping in Jira

    Atlassian has updated its Jira project management software for managing software development initiatives to add more capabilities to road mapping tools that teams use to share plans and update progress.

    Matt Ryall, head of product for Jira Software at Atlassian, said the road mapping tools first provided in Jira a year ago are now being employed by 45% of customers, making it the most rapidly adopted feature in 18 years the project management application has been around. Thus far, Atlassian reports end users have created more than 2 million road map items within Jira.

    Additional capabilities added to the road mapping tools provided in the latest edition of Jira include a drag-and-drop dependency mapping feature, support for filters to track projects more easily, a progress bar that indicates macro-level progress of each road map item, an ability to set up hierarchy levels on the road map, and the ability to set up road maps inline with a single click.

    road mapping

    In addition, a live Jira road map can now be integrated with Confluence, a complementary tool from Atlassian for managing projects spanning multiple teams.

    Atlassian currently offers several road-mapping solutions, including a team-level option in Jira, a program-level road map known as Portfolio for Jira and an enterprise organization-level version dubbed Jira Align. Atlassian has committed to making all three of these offerings work better together in 2020.

    In addition, Atlassian is committing to bringing together the various road-mapping capabilities is provides across its portfolio. The company plans to enable organizations to aggregate multiple road maps into a single consumable artifact and is exploring ways to make road-mapping functionality accessible beyond Jira users. As more organizations in the age of digital business realize how much their future success depends on timely delivery of software, the number of end users who want to be able to monitor the pace of application development projects continues to expand.

    In general, Ryall said Atlassian is focused on modernizing the end user experience for a project management application that is employed by more than 65,000 customers to manage software development projects. Much of that effort is focused on adding a visualization layer that will make it easier to navigate software development projects that continue to become more complex as organizations embrace, for example, microservices-based architectures.

    Competition across the project management category is intensifying as startups look to drive alternative approaches that they claim are designed to support, for example, a mobile-first user experience that makes it easy to visualize data. It’s not clear to what degree any of those offerings are gaining traction, but a significant amount of venture capital is starting to pour into the sector.

    In the meantime, visualization tools that promise to make it easier to appreciate what project events will have the most impact on the business are always going to be welcome. The real challenge, of course, remains getting developers to share accurate assessments of their progress with project team leaders. The good news is, it’s becoming much easier to automatically gather that data from the various repositories developers now rely on to check code in and out of various systems without having to ask anyone for an actual update.

    — Mike Vizard

  • The Risks and Potential Impacts Associated with Open Source

    The Risks and Potential Impacts Associated with Open Source

    Open source software (OSS) is built by communities of developers who contribute their knowledge and time to OSS projects they find appealing. That code can then be used by individuals, communities and organizations in their software products—the only obligation they have is to play under the rules of the license with which the OSS project was published. 

    This type of knowledge sharing brings many benefits to OSS users as it speeds up software development time and can help companies become more competitive in the market. Unfortunately, there is also a catch. Those benefits also come with certain risks which every OSS user needs to be aware of and take necessary actions to mitigate.

    The OSS License

    One specific risk to consider involves the OSS license. Not knowing what your obligations are under the license (or not abiding by those obligations) can cause an OSS user to, for example, lose intellectual property or experience a monetary loss.

    Operational Impact on the Product

    Another risk is the operational impact that the OSS in use will have on the product. It’s important to understand whether the community is consistently managing the OSS project and its issues in a timely manner, and that your organization has implemented the latest version and deployed patches have been applied. Using out-of-date, unpatched or unmaintained OSS could impact your release capabilities, product quality and customer trust.

    Security Vulnerabilities

    The third risk to consider when using OSS is that of security risk. As open source is software—and in many cases very complex software—it brings with it the possibility of introducing software security vulnerabilities into your source code. Some vulnerabilities can be introduced into your projects through dependencies that the OSS your organization is using has on other OSS projects. Additionally, vulnerabilities could be unintentionally introduced by developers as they code a functionality using open source. The reason for this is that software developers, whether professionals or hobbyists, oftentimes do not consider security while coding. This has a substantial impact on their work and their contributions to open source projects. 

    It’s not necessarily that developers intentionally skip the implementation of security procedures. Secure coding is a specific discipline in software development that guards against the accidental introduction of vulnerabilities—a discipline that must be learned. 

    Understand that many developers are self-taught or have been trained in schools and universities (where we are only recently seeing courses in software and application security emerging). Many use their software development skills as a hobby, to contribute to OSS projects or to pursue their own personal projects. Unfortunately, training programs, programming courses in schools and online tutorials often neglect to convey this knowledge or to stress the importance of building secure software. 

    It’s also important to note that many organizations find it difficult to introduce a secure coding discipline into their software development process. In other words, this struggle persists beyond open source projects. If you’re with an organization experiencing this pain point and unsure where to begin, you’ve come to the right place. There are several strategies that can be used to begin your secure coding journey. 

    Training

    By giving developers the education required for them to become security aware, you’re setting them up for consistent, secure development success. There are a variety of in-person and computer-based training options on the market.

    Software Composition Analysis (SCA) Tooling

    Implementing an SCA tool will uncover incorrectly coded strings or vulnerabilities that have been introduced in code or OSS components. Not only can SCA tools assist developers in identifying potential software risks, some SCA solutions can help to train developers by offering remediation guidance or short training videos on how to avoid such vulnerabilities in the future.

    It’s essential for software issues to be discovered early because the deeper into the process they’re identified, the more time-consuming and costly they are to remediate. As such, in recent years there has been a push to shift left, introducing security into the product development life cycle from the very beginning. 

    Companies and individuals need to also understand that this security mindset doesn’t start and end in the coding phase of the development process. A security mindset needs to be a part of every phase: from design to deployment to maintenance, all the way through to the product’s end of life. 

    Emphasizing a security mindset also has a long-term, broad reaching positive effects on the whole software development ecosystem, including the OSS world. As developers are trained to code securely (through university courses and other training institutions, coding portals or professional development through their employer) they pass this knowledge on to the projects with which they’re involved. This then leads to the production of higher quality proprietary and open source software, which in turn minimizes risk across the board.

    — Boris Cipot

  • IBM Open Sources SysFlow Monitoring Platform

    IBM Open Sources SysFlow Monitoring Platform

    IBM has announced that its SysFlow monitoring platform is now available as an open source project.

    Fred Araujo, a research scientist in the Cognitive Cybersecurity Intelligence Group at IBM Research, said IBM developed lightweight SysFlow agent software and monitoring tools as a way to provide more context around the telemetry data being collected while simultaneously reducing the amount of data that needs to be stored.

    SysFlow encodes a representation of system activities into a compact format that records how applications interact with their environment, Araujo said, noting that level of context provides deeper visibility in everything from container workloads to cybersecurity forensics. However, unlike existing monitoring platforms, SysFlow doesn’t require IT organizations to collect a massive amount of data to achieve that goal—it is intended to provide for a superset of the NetFlow framework used to analyze network traffic patterns to capture system events, he said.

    Araujo noted IBM doesn’t envision SysFlow eliminating the need for legacy log analytics platforms, as they provide a way to analyze log data. However, SysFlow does enable IT organizations to apply analytics via a graph-like visualization to surface patterns that goes beyond a comparative simple rules-based approach, said Araujo. For example, SysFlow’s approach will make it easier to uncover the relationship between various events that make up a cybersecurity attack and subsequently to identify what countermeasures to employ to create the appropriate kill chain response. It also should substantially reduce the amount of fatigue cybersecurity teams experience from chasing down false-positive alerts, he said.

    SysFlow is designed from the ground up to integrate with both open source frameworks such as Apache Spark and commercial analytics platforms via an open serialization format and associated libraries. IT organizations also can leverage a set of reusable components and APIs to make it easier to deploy telemetry probes. IT organizations also can take advantage of an extensible policy engine that can ingest customizable security policies described in a declarative input language, which then can be checked against records captured by SysFlow.

    Araujo said IBM developed SysFlow to address the challenges associated with managing application workloads in public clouds, but the framework can be applied to any on-premises IT environment to create a single pane of glass through which multiple platforms can be monitored.

    Regardless of the platform on which a workload is deployed, Araujo said as more responsibility for managing and securing applications shifts left toward developers in the age of DevOps, those teams need access to more sophisticated tools that are readily available and simple to deploy. There are several open source initiatives underway to provide that visibility, and one day many of them might coalesce into a single initiative.

    In the meantime, as the dependencies between applications and IT infrastructure continue to increase, it’s clear IT monitoring as a core part of any set of best DevOps practices is now an absolute requirement to achieve and maintain observability.

    — Mike Vizard

  • Che: A Revolutionary IDE for the Mainframe

    Che: A Revolutionary IDE for the Mainframe

    Tools such as the green screens of ISPF and the Eclipse desktop IDE, enhanced with proprietary plug-ins, have served mainframe application developers well over the years and, for those comfortable with them, will continue to do so. However, changes in the broader world of development are creating the conditions for a revolution in mainframe tooling.

    First, mainframe developers are aging out of the workforce, leaving behind extensive code libraries and a workforce skills gap. Those likely to back-fill these roles know and love modern IDEs, especially Visual Studio Code, while the popularity of Eclipse is waning. For example, the annual Stack Overflow Developer Survey found that, over the past year, the popularity of Visual Studio Code grew from 35% to 51%, once again making it the most popular IDE, while Eclipse fell from 19% to 14%. According to Business Insider, 8.5 million developers worldwide use VS Code, creating a huge talent pool.

    Second, application development has made great strides in productivity since the current mainframe dev tools were created, especially in DevOps automation with enablers like task runners, scripting and testing frameworks. The Stack Overflow survey cites scripting language Python as the most sought-after skill among developers, by far.

    Finally, as the velocity of overall software delivery increases, mainframe is a critical component of digital transformation initiatives. According to 451 Research, 24% of companies are releasing application software daily or hourly while, similarly, DORA’s 2019 State of DevOps survey shows 20% of teams are deploying multiple times per day. Software delivery expectations have changed with continuous deployment becoming the new normal and, to remain a vital computing platform for the long term, mainframe app development needs to support this paradigm.

    An Open Source and Cloud Development Technology 

    Rather than an evolution of the existing Eclipse IDE, Eclipse Che is a radical departure. According to the Eclipse Foundation, “Che 7 is the world’s first Kubernetes-native IDE that’s built from the ground up for cloud native application development. It simplifies and accelerates cloud development by allowing developers who are not Kubernetes experts to immediately contribute to cloud native application development efforts. In turn, Che 7 relies upon Eclipse Theia, which provides a highly modular and extensible IDE platform built on modern web technologies that runs on both your desktop or in your browser.”

    For large enterprises, a good analogy for key value proposition of a hosted IDE is Git: usability and collaborative power have made Git the de facto standard for version control and Che’s workspace server offers comparable benefits.  

    Developers working on mainframe applications can use Che via the Che4z subproject. Che4z, whose primary goal is to bring a modern developer experience to mainframe software development, offers an extension pack providing access to the z/OS file system, remote debug and remote editing of COBOL programs with all the expected modern editor capabilities including syntax highlighting, code assist and real time syntax validation. After a beta period, Che4z is now GA.

    Beyond the foundational benefits mentioned above, Che is revolutionary for the mainframe for many reasons.

    Visual Studio Code and More

    Che was architected around a VS Code-like experience from the outset which, with the growth of VS Code, means Che extends its value.  Che is based on Eclipse Theia which provides an in-browser VS Code experience complete with the latest tooling protocols: language server, debug adapter and compatibility with VS Code extensions. 

    The Che4z sub-project, contributed by Broadcom, provides language support for both COBOL and High Level Assembler (HLASM).  The language server protocol (LSP) enables developers to use these languages with Che, VS Code and any other LSP clients (e.g., IntelliJ, VIM, Emacs).

    Bottom line: developers can use VS Code for mainframe development independent of Che adoption (the Code4z extension pack is available on the VS Code marketplace).

    Cross-Platform Applications

    For many Fortune 500 companies, the mainframe hosts critical system-of-record applications and data while the cloud and mobile applications have evolved from user-oriented systems-of-experience. With tech-first startups disrupting entire industries, large companies are deploying more cross-platform applications to maintain, if not win, market share by harnessing the best of both worlds. Cross-platform apps combine modern user experiences with the transactional power and data assets of mainframes.

    Because cloud and mobile developers need to integrate mainframe applications, allowing them to use the tools they know and love is critical. This includes the popular code editors and IDEs they love today as well as the new innovations, such as browser-based, container deployed tools that make them more productive.

    With its workspace server, Che also facilitates pair programming, which is a powerful approach when mainframe and distributed developers need to collaborate on cross-platform applications.

    Bottom line: Che makes the mainframe less of a silo and more like other platforms.

    Open Source-Enabled Vendor Independence

    While open source is not new to the mainframe–zLinux has been around for years–access to off-platform toolchains became possible with the introduction of the Zowe open source framework (described in more detail below). Unlike the incumbent Eclipse vendors, and consistent with the open source vision, the Che4z extensions are open sourced and free to use. There is now an alternative to the proprietary, vendor-controlled model: true open source tooling for mainframe developers.

    Furthermore, Eclipse locked mainframe developers into using a specific set of editors, runtimes and user interfaces for required capability and functionality–thus imposing a steep learning curve. Che, by comparison, offers flexibility via modular design, allowing individual developers to choose the components they need.

    Whether developing for mainframe or other platforms, the Che community welcomes contributions from those passionate about software development. Want to incorporate a lesser-used language? No need to plead with a vendor. Anyone can seek community support for an idea or simply do it themselves. The Che/Che4z communities will ensure the proper procedures are followed before any code is ever released.

    Bottom line: Che provides a more frictionless, fast-moving toolset allowing coding innovations to extend to the mainframe.

    Onboarding and Cost Reduction

    Consider the compatibility and maintenance headaches created by the heavyweight plugins required for Eclipse. For mainframe leaders, the need to reduce costs is an ongoing challenge as is the need to quickly onboard the next-generation of developers. As a container-based IDE with no client footprint, Che eliminates the need for local installation, maintenance and configuration. The potential of moving entire teams currently using Eclipse with customizations to Che provides an attractive opportunity to materially reduce operating costs while simultaneously improving developer productivity.

    Because Che workspaces are containerized, new team members can onboard instantly–open the container, begin work. This means developers, who are high cost resources, can more easily move across projects, teams, departments, etc.

    Bottom line: For mainframe leaders focused on staffing and costs, Che increases the talent pool while eliminating costs.

    Built on Zowe

    The extensions that open the mainframe to the Che IDE are powered by the Open Mainframe Project’s Zowe open source framework, which maintains mainframe-native security standards. Zowe, which was founded by Broadcom, IBM and Rocket Software and is the first open source project based on z/OS, enables developers to manage, control, script and develop on the mainframe just like cloud and other platforms. 

    Command line interfaces (CLIs), which come with platforms such as AWS and Azure, are popular with developers wanting the productivity lift of task runners, scripting and higher levels of automation (i.e., CLIs facilitate DevOps adoption). The Zowe CLI (the mainframe equivalent) magnifies the value of the Che4z stack as any set of commands that run through the CLI can be automated.

    Bottom line: Using Zowe as the foundation for the Che4z mainframe extensions ensures the modern experience extends far beyond the IDE.

    Path Forward

    For leaders wanting to empower developers integrating mainframe code and resources with the most popular and powerful app development tools, Che provides the best of both worlds–the immediacy of VS Code use with the path to full-on, in-the-cloud team collaboration.

    Expanded tool choice, greater alignment with off-platform peers and DevOps-fueled software delivery are all reasons why Che is not only the first Kubernetes-native IDE, but a real revolution for mainframe development.

    — Jean-Louis Vignaud

  • SaltStack Extends IT Automation Framework

    SaltStack Extends IT Automation Framework

    SaltStack today added three open source modules to its portfolio that make it easier to automate the management of IT at scale.

    Heist is a plugin that allows organizations to create agents that will dissolve after a specific amount of time. Umbra is a plugin that gathers data streams to make it easier for machine learning algorithms to consume.

    Finally, Idem is a declarative programming language that enables IT teams to create workflows based on stateful constructs that can be used in conjunction with any IT automation framework or systems management tool.

    All three modules are based on the Salt Plugin Oriented Programming (POP) model. Company CTO Thomas Hatch said SaltStack created POP to make it easier to manage modules and libraries of code within a monolithic application without having to transition to a different microservices-based architecture based on containers.

    That POP approach also serves to make it easier to test modules within a monolithic application environment, he said.

    SaltStack used POP to create Salt, an open source IT automation framework that serves as the foundation for the commercial IT automation platform provided by SaltStack. Organizations that have embraced the open source Salt project include Juniper Networks, Cloudflare, Nutanix, SUSE, LinkedIn and eBay.

    Heist extends the Salt project by giving DevOps teams an alternative to having to commit to either an agent or agentless-based approach to managing IT. Instead, DevOps teams can create an agent to collect data that will run only as long as it’s needed.

    Umbra, meanwhile, prepares data to be consumed by machine learning algorithms transparently, which Hatch said is critical in providing visibility into how artificial intelligence (AI) models are constructed. That capability is going to be a requirement in any regulated industry that requires organizations to comply with IT audits, he noted.

    IT automation frameworks are, of course, foundational to any set of best DevOps practices. While most of the initial IT automation management frameworks were proprietary, the entire category has gained more traction as open source implementations of these frameworks became available. Now DevOps teams have a plethora of open source and commercial IT automation frameworks on which to define a set of DevOps processes.

    The biggest challenge in getting IT teams to embrace these frameworks historically has been the preference of many IT professionals for scripts they develop. However, as the scale of the IT management challenge has increased in the age of the cloud, relying on scripts that are often undocumented has become problematic. Industrial-scale IT organizations require IT automation frameworks that allow them to create reusable playbooks that are easily documented. Otherwise, every time an IT professional who authored a script leaves the company, the IT organization soon discovers the script they’ve been relying on that has broken can’t be fixed easily.

    The day when all IT organizations rely on some form of an IT automation framework is fast approaching. In fact, the remaining IT automation challenges have a lot more to do with the culture within the IT organization than they do any of the technical capabilities of the platforms being considered.

    — Mike Vizard

  • Perfect Sense Open Sources Gyro to Automate Cloud Provisioning

    Perfect Sense Open Sources Gyro to Automate Cloud Provisioning

    Perfect Sense has launched an open source project dubbed Gyro that provides an alternative approach to automating cloud infrastructure provisioning and management.

    Gyro was developed as an alternative to other open source projects such as Terraform and Pulumi, said CTO Jeremy Collins. It is designed from the ground up around a programming language that allows DevOps teams to define infrastructure as code, and features logic built into configurations to make it easier to create more precise workflows, he said.

    Adding logic to create equivalent workflows in rival platforms is more complex, Collins noted, adding the ability to add logic makes it easier to not only provision cloud infrastructure but also to roll it back when necessary.

    Perfect Sense developed Gyro to make it easier to manage the infrastructure for its Brightspot content management system (CMS). As such, Gyro can support multiple cloud computing platforms.

    In addition to making it possible to add cloud platforms to an existing configuration, Collins said it can be used to create custom commands and define multiple extension points directly within the Gyro language.

    Perfect Sense Gyro

    Perfect Sense developed Gyro to automate the provisioning of virtual machines. Since then, the company has extended its capabilities to add support for emerging cloud-native platforms such as Kubernetes.

    Collins said Gyro provides a single extensible tool spanning a broad range of DevOps processes, from provisioning infrastructure to discovering what services already exist.

    Despite the popularity of rival tools such as Terraform, it’s still early days in the adoption of infrastructure as code. Most IT organizations are a long way from standardizing on any one platform for automating the provisioning of infrastructure. In fact, it may not be uncommon for different DevOps teams within the same organization to employ different platforms for provisioning infrastructure.

    Gyro’s ultimate success will depend much on Perfect Sense’s ability to foster an open source community. The Brightspot CMS is used by the Los Angeles Times, Walmart, Mattress Firm, Univision, Sotheby’s and US News and World Report, so Collins said there is a base of support for Gyro on which to build a community. Perfect Sense is looking to build a community that will continue to extend the reach and scope of the platform by adding additional plugins.

    Regardless of the approach taken, as the number and rate at which workloads are being deployed on cloud platforms increases, it becomes critical for IT organizations to find a way to automate processes. It’s not clear to what degree those workloads will be provisioned by developers themselves versus traditional IT operations teams that have learned how to program. However, one way or another the provisioning of cloud infrastructure is going to be automated.

    — Mike Vizard

  • Building a Real-Time App: 13 Open Source Projects You Should Be Tracking Right Now

    Building a Real-Time App: 13 Open Source Projects You Should Be Tracking Right Now

    Open source is the dominant model of software consumption in the modern era. Cutting-edge startups and entrenched incumbents alike find open source software development and community building a significant part of their overall business strategies.

    For many open source projects, independent steering committees ensure that projects remain stable and are not beholden to a single company or profit motive. Alternatively, for open core projects where a single company financially supports and contributes the vast majority of open source code, open sourcing makes a project publicly available and accessible for potential new users. Opening a project to public contribution widens the pool of potential users and contributors, therefore ensuring more rapid innovation than commercial software offerings.

    Choosing the Right Open Source Projects for Your Real-Time App

    There are seemingly limitless open source projects to evaluate. Further complicating matters, each project has its own domain competence and cultural nuances. Considering a single distributed application will have tens or (many) more open source dependencies, development teams must continuously test new open source offerings and hone new skills to use these solutions, while simultaneously architecting their applications.

    In order to help sort through the noise, I’ve looked at six categories of open source projects for this article: data storage, message systems, service meshes, REST frameworks and streaming frameworks. For each category, I’ll identify any dominant players as well as some other projects of note.

    Here are the top 13 open source projects by category.

    Data Storage: Cassandra, Hazelcast

    Apache Cassandra, originally developed at Facebook, is one of the most widely used open source projects for real-time applications. Functionally, it’s a distributed NoSQL database management system. But practically, it’s become the linchpin of most at-scale, real-time applications. This is because Cassandra was designed to handle large amounts of real-time data across many servers, and has robust clustering support.

    For applications that require extremely low latency or have exceptionally high data rates, Hazelcast may provide a useful alternative. Unlike Cassandra, Hazelcast is an in-memory data grid database, which enables development teams to benefit from the latency advantages of in-memory computing. 

    Message Systems: Kafka, Pulsar, NATS

    Apache Kafka is the dominant open source messaging system used by application development teams today. Originating at LinkedIn, Kafka has grown into the de facto standard for high performance open source messaging systems. For streaming use cases, Kafka users can also turn to Kafka Streams.

    Apache Pulsar, developed at Yahoo!, is another alternative messaging system which has been optimized for streaming data sources. Pulsar further differentiates itself with TTL message support and queuing via shared subscription.

    For applications where latency is a critical consideration, NATS.io (a CNCF project) is one of the fastest pub-sub messaging systems available.

    Service Meshes: Istio, Linkerd

    Trends such as microservices, serverless and hybrid cloud deployment have contributed to the increasing architectural complexity of distributed applications. With hundreds or thousands of distributed application services, service meshes provide a mechanism for pulling all the disparate services together into a manageable and cohesive whole.

    The most notable open source service mesh offering is Istio. One of the most popular open source service mesh projects, Istio is primarily used for applications built on top of Kubernetes. Another alternative is Linkerd, a flexible service mesh solution for Kubernetes or other frameworks.

    REST Frameworks: Node.js, Spring

    Node.js is a popular JavaScript framework for building scalable web applications. When comparing web development frameworks, Node.js typically ranks among the top three most (along with frontend frameworks React.js and Angular.JS). While Node.js can support large scale applications, it does not support real-time use cases well at scale.

    Spring is another application framework, optimized for distributed microservices-style architectures. Like Node.js, Spring “focuses on the ‘plumbing’ of applications so that teams can focus on application-level business logic, without unnecessary ties to specific deployment environments.”

    Streaming Frameworks: swimOS

    SwimOS is a software framework for distributed streaming applications. Unlike REST-based frameworks, swimOS utilizes a unique and lightweight Web Agent architecture, optimized for composing and transforming many data streams in real-time. Whereas DevOps oriented projects, such as Spinnaker, seek to provide an abstraction layer to converge distributed cloud services, swimOS provides an abstraction across distributed data streams, so teams can build applications by composing many distributed data streams in a single mesh environment.

    In order to ensure real-time consistency of distributed services, swimOS automatically ensures eventual consistency between all Web Agents. Complete with a UI toolkit and it’s own stateful messaging protocol called WARP, swimOS is the fastest way to build stateful streaming applications.

    DevOps: Kubernetes, Spinnaker, RancherOS

    Kubernetes is one of the most talked about open source projects in the world. Kubernetes describes itself as “an open-source system for automating deployment, scaling and management of containerized applications.”

    While it may simply be a platform for container orchestration, rightly or wrongly, Kubernetes has become a stand-in for DevOps on the whole. In spite of its dominance, there are alternatives to Kubernetes, for example there’s RancherOS to manage Docker containers.

    Furthermore, as application architectures continue to become more distributed and complex there is a need for platforms which can converge multiple distributed edge and cloud environments. Spinnaker provides an abstraction layer to connect the various environments and provides a simple mechanism for continuous integration/continuous delivery (CI/CD). 

    Final Thoughts

    Your own particular use case should determine the appropriate open source technologies for your team’s real-time application. Whether you’re building an ultra-high performance low latency application or simply trying to tame large volumes of real-time data, knowing your options and choosing the right framework or platform can save you significant time and effort.

    The open source software landscape is changing every day. If you have something to add about one of the technologies above or think I skipped over the next OSS breakthrough, I’d love to hear about it. Let me know your thoughts in the comments section below.

    — Chris Sachs