Tag: ci

  • GitLab Expands Scope of DevOps Ambitions

    GitLab Expands Scope of DevOps Ambitions

    GitLab, with the release of its 12.0 update, is extending the reach of its namesake continuous integration/continuous delivery (CI/CD) platform to include both cybersecurity professionals and traditional IT operations staff.

    John Jeremiah, senior product marketing manager at GitLab, said the goal is to provide organizations with a single CI/CD platform that unifies the management of everything from source code to best DevSecOps processes.

    To address that latter requirement, GitLab 12.0 adds a dashboard designed specifically for security personnel along with the ability for cybersecurity teams to approve merge requests and automatically apply patches for vulnerabilities detected. Those capabilities complement existing support for scanning and secrets detection.

    New capabilities embedded in the platform for IT Operations professionals include an operations dashboard and an embedded incident management capability that makes it easier to create a coordinated response to a specific event.

    GitLab 12.0 also adds support for feature flags to better manage the application delivery process, along with a platform-as-a-service (PaaS) environment to provision and maintain the infrastructure needed for developing and launching an application.

    Jeremiah said the new and existing capabilities in GitLab will make it possible for organizations to unify their DevOps tool chains in a way that also enables IT operations teams and cybersecurity professionals to be included in the process.
    At the same time, GitLab is making DevOps more accessible by eliminating many of the silos between various tools that have emerged over the years, including, for example, built-in support for feature flagging versus requiring organizations to acquire and maintain a separate tool, he noted.

    Because those capabilities are embedded in the platform, organizations will be able to make the cultural shift required to adopt DevOps more easily, he added. IT teams don’t always know what other tools they will need as their DevOps skills mature. With all the tools embedded in GitLab readily available, it becomes easier for organizations to expand their DevOps processes as the organization starts to implement more sophisticated processes, Jeremiah said.

    That approach is starting to resonate with the 100,000-plus developers actively using the GitLab platform and more than 2,000 contributors actively contributing code to the underlying open source platform, he added. GitLab in March saw its millionth merge request.

    It’s too early to tell how broadly any CI/CD platform might be able to consolidate the DevOps toolchain. Developers are notorious for adopting new tools as they see fit. Internal IT teams, of course, are not always excited to be asked to support yet another tool. The right level of balance between ultimate flexibility and the need for a standard set of processes that can be implemented across multiple application development teams must be determined by each organization.

    In the meantime, DevOps processes are starting to expand beyond traditional development team to now include IT operations staff and cybersecurity personnel. The challenge now that they’ve been invited to the proverbial DevOps party is making them feel welcome and appreciated.

    — Mike Vizard

  • CloudBees Allies with Atos to Accelerate DevOps Adoption

    CloudBees Allies with Atos to Accelerate DevOps Adoption

    In a move that should accelerate adoption of best DevOps practices in the enterprise, CloudBees and Atos announced this week that the two are collaborating to advance adoption of continuous integration/continuous development (CI/CD) platforms on the Google public cloud.

    Announced at the Google Cloud Next 2019 conference, the alliance will result in the IT services giant delivering a managed instance of a CloudBees CI/CD platform running on the Google cloud, in addition to providing traditional consulting services.

    Francois Dechery, chief strategy officer for CloudBees, said the alliance with Atos will accelerate adoption of DevOps processes in traditional enterprise IT organizations as Atos consultants engage customers looking to modernize their application development and deployment processes. In fact, many of the principles first pioneered by proponents of agile development methodologies are now being applied to business process management, he noted.

    Atos plans to add a DevOps practice as a complement to its existing processes. Michael Kollar, senior vice president for cloud engineering, said the company expects that, for the foreseeable future, most of their clients will be trying to meld some degree of DevOps and existing ITIL-based approaches to IT services management (ITSM). The alliance with Google will go a long way to accelerating that transition because it will allow Atos to deliver access to a CI/CD platform based on a consumption model that reflects the way public cloud infrastructure is consumed, he said.

    Kollar noted enterprise IT organizations are under more pressure than ever to deliver applications faster as more organizations recognize they are a software company that happens to make something. Most organizations today need to create a superior customer experience through software to differentiate themselves from their competition, he said.

    Many of those companies will need help setting up the processes that will enable them to function as a software company, he added. The biggest challenge, however, is finding a way to meld new ways of building and deploying applications with investments in legacy systems on which the business still depends, he said.

    Naturally, Atos won’t be the only IT services firm to launch a managed DevOps process. But as more IT services firms invest in developing managed services, there will be need for a balance between the managed services provided by IT services providers and cloud service providers such as Google and the IT services that will continue to be managed and delivered by internal IT organizations. The degree to which organizations will rely on external IT services firms for DevOps will be dictated by how mature they are today. But it does not follow that an organization which sees a need to rely on external expertise today will be that position tomorrow. As more organizations develop internal expertise, a managed service provider will need to earn its seat at the table every time its contract comes up for renewal.

    — Mike Vizard

  • The Linux Foundation Launches Continuous Delivery Foundation

    The Linux Foundation Launches Continuous Delivery Foundation

    The Linux Foundation announced it will provide the home base for a vendor-neutral Continuous Delivery Foundation (CDF) committed to making it easier to build and reuse DevOps pipelines across multiple continuous integration/continuous delivery (CI/CD) platforms.

    The first projects to be hosted under the auspices of CDF, which was launched at the Open Source Leadership Summit conference, includes Jenkins, the open source CI/CD system, and Jenkins X, an open source CI/CD solution on Kubernetes. Both were developed by CloudBees. Netflix and Google, meanwhile, are contributing Spinnaker, an open source multi-cloud CD solution, and Google is also adding Tekton, an open source project and specification for creating CI/CD components.

    Additional projects are expected to join CDF once a Technical Oversight Committee (TOC) is formally named.

    Founding members of the CDF include Alauda, Alibaba, Anchore, Armory, Autodesk, Capital One, CircleCI, CloudBees, DeployHub, GitLab, Google, Huawei, JFrog, Netflix, Puppet, Red Hat, SAP and Snyk.

    Chris Aniszczyk, vice president of developer relations for The Linux Foundation, said that while a lot of progress has been made over the years regarding adoption of CI, the challenges associated with mastering CD have proven more vexing. To address that issue, the CDF will evangelize CI/CD and DevOps methodologies, define and document best practices, provide guidelines and create training materials to better enable organizations to implement CI/CD best practices.

    Kim Lewandowski, a product manager for Google Cloud, said making it possible for organizations to reuse DevOps pipelines is a high priority. Building pipelines is a significant challenge for most DevOps teams; as such, they need to be able to reuse those pipelines across as many CI/CD platforms as possible.

    Kohsuke Kawaguchi, creator of Jenkins and CTO at CloudBees, noted the creation of interoperability standards between CI/CD platforms will go a long way toward achieving that goal.

    It’s not clear yet just how widely the CDF will be supported by the entire DevOps community. While CloudBees provides one of the most widely employed CI/CD platforms, other offerings have started gaining traction, especially among enterprise IT organizations that tend to not have a lot of internal DevOps expertise.

    On the plus side, the CDF should help spur adoption of DevOps best practices even further. Adoption of DevOps processes within most enterprise organizations tends to be uneven at best. One team may be employing DevOps extensively, while the rest of the organization continues to rely on less agile legacy processes.

    The CDF also should play a significant role in enabling organizations to embrace microservices-based applications built using containers. While Jenkins is the most widely relied-on CI/CD platform today, it is built on what is now considered in many quarters to be a legacy Java Virtual Machine (JVM) platform. In contrast, Jenkins X Background an open source CI/CD platform for building and deploying modern cloud-native applications on Kubernetes. The Linux Foundation also plays host the Cloud Native Computing Foundation (CNCF), which oversees the development of Kubernetes.

    It still may take years for IT organizations to fully make the transition to DevOps and cloud-native applications. But in the months ahead, it’s clear the CDF intends to function as a force multiplier in enabling organizations to achieve that goal.

    — Mike Vizard

  • How Mainframes Live On

    How Mainframes Live On

    TL/DR: Highly controlled change is sometimes the best answer.

    For much of my career, I’ve worked in mainframe shops. I have never worked on a mainframe team, but I have worked closely with them. I’m in Northeast Wisconsin, and at one point we had the highest per-capita number of mainframes in the world, so pretty much anyone from this area can say the same. They were the “stable core” of the system for which I was the “new shiny”—and at times, that was painful.

    If you’re an established enterprise, even after years of trying to eliminate the mainframe, you probably still have one. They don’t go away so easily because, despite frustrations with rate of change, they do host core systems in established organizations and are reliable. I have seen similar in client server systems that are core to the business.

    We are reaching a point where there are some great products out there to help modernize the mainframe and/or split up client/server applications. Some of them have been around a while (Linux) and are coming into wider adoption, some of them are building on that bit of more familiar interface and many are simply making the UI more familiar to developers like me that only ever developed in a mainframe terminal for college, if at all.

    One of the main reasons why mainframes and future iteration’s applications are “sticky” is because they, and the systems they host, are generally reliable. No matter what you think of the timeline for updates, the data coming out of the mainframe and more controlled distributed applications is considered accurate in most organizations, over most other sources.

    To be clear, there are groups like this from each successive wave of computing. The client/server app that replaced your mainframe customer information system has learned over time that a fault in their system impacts the core of the business, for example.

    The goal is to increase the rate of change, without undermining the reliability. You hear a lot about throw-away instances and “fail fast,” but dropping your billing system isn’t really an option, so these need to be tempered with the knowledge of what failure implies in core systems.

    Make the change, do the modernization, but listen to lessons learned on core systems. Take into account that your informational website being down is a serious problem for marketing, but the billing system being down is a disaster for the company. There is a degree of difference, and the approach should be less about continuous deployment and more about stable deployment. Yes, it can be a scale, and you can start the motion from one to the other slowly to make sure you’re covering your bases.

    In short, assess the risk levels of these systems before deciding how to proceed. This is pretty much common-sense advice—but better to read it and not need it, than to find out when corporate revenues are impacted.

    I worked at one shop where a certain core application was using technology that was a couple of generations old and no longer supported. The pool of available developers was shrinking, but it was an important application. This type of thing is where modernization is critical; having someone say, “We don’t touch that,” is no longer acceptable. That story ends in disaster when the system fails, and no one knows how to troubleshoot—let alone fix—it.

    As with so many things, the balancing act is up to you; determine your organization’s level of risk acceptance, then determine the impacts of this application being unavailable and proceed from there.

    Each step of mainframe development is a little different from what developers coming out of college expect, but there are new tools and mechanisms available of which these developers can take advantage.

    — Don Macvittie

  • Stop the Domino Effect by Joining Build Engineering and DevOps

    Stop the Domino Effect by Joining Build Engineering and DevOps

    As DevOps has become more common, these teams are tasked with uniting everything from coding to deploying and monitoring applications. Build Engineering forms the foundation for all those processes and is essential for building code. Its tools vary according to the operating system you’re building on and by programming language. DevOps needs to work with Build Engineering, but the two don’t usually connect. And that’s a problem.

    Over the last 10 years, there’s been a shift toward DevOps. Previously there was a discrete division between coding, done by developers and engineers, and deployment, done by Ops. However, it became apparent that for companies to optimize benefits such as productivity, development and operations shouldn’t be so disparate. The result of which is DevOps.

    Along with the shift to DevOps have come several trade-offs—namely trading security for speed, certainty for agility and proprietary code for open source:

    • Speed: Getting to market quickly has become critical to success. Speed now takes greater priority over fixing vulnerabilities, which can always be addressed in the next release (hopefully). And retrofitting for security and vulnerabilities after the fact becomes a big blocker for development and engineering teams. At the end of the day, the risk of lost profits is greater than the risk of potential unknown threats.
    • Agility: Agility trumps certainty in the race to market. For instance, iterative pushes to production do away with fixed, known road maps in favor of implementing whatever this week’s most important feature/functionality is. The trend toward immutability is also important here; it’s generally simpler and quicker to throw away something that’s broken or outdated rather than trying to fix or update it.
    • Open source: Due to the greater innovation, reduced costs and transparency it delivers, open source has won over proprietary code. Rather than focusing on creating proprietary code, developers can quickly iterate on a solution by assembling open source code, accelerating time to market. Plus, the ability to take from what has already been created can free up your dev teams to innovate and build IP for your enterprise efforts.

    For most, the tension between fixing something and knowing if it will break something upstream hasn’t been eliminated. You may be able to quickly update and patch a single solution, but how will this impact you once you push to your CI stream? Even for mature DevOps organizations, there’s a hidden opportunity cost associated with two factors that are overlooked: open source programming languages and Build Engineering.

    Overhead is added to your DevOps teams when you retrofit your open source language of choice with new versions, dependencies, security patches and so on adds. Teams should be focused on the things that matter—creating and delivering the latest release.

    On top of that, Build Engineering takes a hit each time the underlying open source language is retrofitted. You need to rebuild all your developer environments, CI/CD systems and production environments. All of this build engineering work is manual, slows down time to market and eats up valuable engineering resources. Plus, you’re not certain your rebuild and update will provide performance improvements that warrant the update.

    Since DevOps already relies on automation, it makes sense to apply this tool to the retrofitting and building of your open source languages and seamlessly connect to the rest of your DevOps cycle. You could ensure build reproducibility, dependency management and compliance with your corporate security and license criteria. For example:

    • Build impact –  What if you could understand the performance improvements (if any) that could be gained by taking the latest build?
    • System update – Consider the benefits of being able to update all of your impacted systems across dev, test and even production environments with the latest build using a single command.
    • Packaging – If each continuous build could be packaged in multiple ways—any form factor for any operating system—how would it change your workflow?
    • Build validation – Imagine that every continuous build could be vetted against your smoke test, inclusive of compliance with your security and license criteria, so you knew whether it was ready to deploy.
    • Continuous builds  – What if every time a new version of a library or patch for a vulnerability was found, a new build was kicked off and you were notified when it was done? Then you could keep up with your continuous deployments.

    The domino effect is a phenomenon that DevOps must deal with on an all-too-often basis. Dev and Ops came together for greater productivity, but unforeseen dependency issues regularly gum up the works. That’s the reason automation is such a useful tool, one that you can used to stop the domino effect as you automate your build pipeline for open source languages. It’s truly possible today to benefit your business by ensuring build reproducibility, dependency management and compliance with your corporate security and license criteria.

    — Bart Copeland

  • DevOps Chat: CI/CD with the Semaphore Platform

    DevOps Chat: CI/CD with the Semaphore Platform

    As the old saying goes, “speed kills.” When it comes to development, a lack of speed can kill, too. Marko Anastasov and the Semaphore team have strived to make Semaphore the fastest continuous integration tool in the market. With version 2.0 of Semaphore, they believe it truly is.

    With his background in development, Marko brings a unique perspective in leading the Semaphore team. He knows what he and his colleagues like, and that is the road map for Semaphore. Have a listen to Marko tell us in his own words.

    As usual, the streaming audio is immediately below, followed by the transcript of our conversation.

    Transcript

    Alan Shimel: Hey, everyone, it’s Alan Shimel, devopsy.kinsta.cloud, and you’re listening to another DevOps Chat. In today’s DevOps Chat, we’re gonna talk about a company called Semaphore. Their website is semaphoreci.com. And I’m happy to be joined today by one of their cofounders, Marko – and I’m not gonna pronounce his last name well, so I’m gonna let him pronounce it himself. Marko, your last name?

    Marko Anastasov: Anastasov.

    Shimel: Anastasov. Correct?

    Anastasov: Yes, that’s correct. Hi, Alan, and thanks for having me. It’s a pleasure.

    Shimel: Thank you for joining us. So, Marko, before we jump into Semaphore, let’s give our listeners a little bit of background on your – who is Marko Anastasov? How did you get here today?

    Anastasov: Yeah. Yeah. Sure. So my core is, you know, I’m a programmer. Big fan of making – I’m a product-guy-slash-programmer. Maybe that’s kind of the intersection where I’m at, so I like making software products. Been doing that kind of almost since I was a kid.

    Shimel: Mm-hmm.

    Anastasov: And the kind of when I started being, let’s say, a professional is when things were going fast on rails, like Ruby on Rails, you know, Web development. And so I got into that, despite being kind of C++ nerd myself, originally.

    Shimel: All right.

    Anastasov: And I just kinda fell in love with ease at which you can kind of make something, put it online, create something useful for other people. I did a couple of kinda noncommercial projects and started a small consultancy, with my cofounder Darko, who is also a cofounder of Semaphore now. And, you know, we’re both also big fans of clean code, doing things – you know, software craftsmanship, all that kind of stuff. And, as we kind of grew ourselves professionally in that whole area, it wasn’t long till we, of course, realized the benefits of processes like, you know – it wasn’t called, really, “DevOps” at that time, ten years ago or something, but we identified, “Okay, continuous integration is obviously a process that any kind of – if you wanna keep moving fast, as you kinda work on a project, this is what you gotta do.”

    And so, through kind of adoption of these practices in our own company and kinda having an eye on an ambition to create a tool that other kind of developers will also find useful, we started prototyping and eventually launched Semaphore in 2012. And, from that point, it’s kind of all we do and it’s been a great ride so far, you know, leading all the way to today’s interview with you.

    Shimel: Sure. Hey, man, thank you. You know what? It’s funny; it’s not easy, but, when you ask people to tell them about themselves, you don’t wanna blow your own horn too much, but you wanna tell people what’s going. And I think you did a good job with it. So let me – before we jump into Semaphore, you mentioned something that I’ve heard from a lot of other developers, people in the DevOps space, whether they be vendors, practitioners, what have you, and that is they were doing what’s called a “DevOps-like things” before we called it “DevOps.” You know what I mean? And so were you into Agile? Was that something that kinda went with that?

    Anastasov: Yeah, I would say yes. Although I never – it was always – Agile itself has always been a bit of a vague concept for me. The first maybe introduction to that kind of thought space, for me, was the “Extreme Programming Explained,” the book by Kent Beck, which –

    Shimel: Sure.

    Anastasov: So it was more like a practitioner’s path for me, so, as you’re in development, working in small iterations, you know, you’re breaking things down, being ready to adapt to change, kind of not being married to some kind of roadmap, and so on. So it was definitely something that we identified as a good way to operate, but even before maybe the Agile backslash – I don’t know. I didn’t use the word “Agile” myself that much, but –

    Shimel: Well, you weren’t doing Scrum or taking Scrum classes and stuff.

    Anastasov: Not really.

    Shimel: But, certainly, sort of the Agile way of doing software development, right, before we called it “DevOps” or anything else, was kinda ingrained in you, if you will. So then I always – and this is something, Marko, I’ve asked other people about – so one of the unique things about DevOps is we sort of build on that Agile, we build on Lean, but, certainly, ops gets involved.

    Anastasov: Yeah.

    Shimel: When did you kinda make that jump, as sort of a programmer/developer, to say, “Hey, it’s really important we get the ops folks, the ops functionality, kinda built in there”?

    Anastasov: So –

    Shimel: And I don’t wanna put words in your mouth.

    Anastasov: Yeah, sure. So, for me, it was about – so I come from a kinda small company environment and that was kinda my beginning. And so, in that kind of setup, if you’re kind of being responsible for a product, means that you’re also responsible for the operational stuff or that there is maybe a shared responsibility among developers to some degree, but there’s always somebody who kinda specializes, at least for a specific project, for doing the operations. And there is – I think it’s very healthy, kind of, for a programmer to kinda be exposed to these things because you – as you kind of experience this firsthand, you kind of  – let’s say you’re ready to – you’re not maybe – you don’t have patience that much for things that kinda get in your way of getting the job done.

    So, if you kind of practice not repeating yourself in writing code, you wanna kind of practice the – try to practice the same thing, when you kinda deal with infrastructure. So I think that’s kind of the line of thought that brought us the kind of infrastructure-as-code, you know, kind of EDU and the movement and the tools force.

    And, once you maybe grow out of that kinda maybe very small team kinda environment, I think what you see then is you see maybe the bigger picture of, if you think of continuous delivery as a broader objective of being able to deploy anything that you’re working on at any point in time, as the ideal state that you wanna be in, then being really good at ops means that, for example, if you’re doing microservices, you’re not really that even Agile in the ’90s, maybe, way, if it takes two weeks for somebody to provision you the resources so you can just get started and write the first lines of code in your microservice. So if that can be reduced to five minutes or an hour or something, that’s really great, so.

    Shimel: Yeah.

    Anastasov: Yeah.

    Shimel: So this is something, Marko, again, back when I first started devopsy.kinsta.cloud, four, five years ago, there was this big debate over “Is DevOps different at a small company versus at an enterprise?” And, of course, there’s differences between working at a small company versus an enterprise, right? In small companies, you do wear multiple hats. You don’t have the luxury of having a full ops team and a full separate dev team and QA and business architects and all that kind of – you know, you don’t have that. Everyone pitches in and does what they gotta do, where, in large enterprises, you sometimes do have that stuff.

    And so, obviously, there’s a difference between them, but, at the end of the day, DevOps is DevOps, right? It’s about doing continuous integration, continuous delivery, and so forth. So I think that’s a – you know, what you’ve described is very not out of the ordinary. I think that’s sort of the path, the track that these things follow.

    Anastasov: eah. Yeah. I think it’s about – whether you’re coming from a small or you’re working at a big organization, ultimately, it’s all about “How do we move fast so that we just kinda of – ” the ideal state is when we’re all kind of in creation mode, all the time or as much as possible.

    Shimel: Sure.

    Anastasov: And it can happen in a small team as much as, I think, in a large team, that things can get stuck, if you maybe – I don’t know – pick the toolset that’s maybe not fit for your needs, where you kind of want to reinvent the wheel or – so I think that, regardless of maybe the team size or the technology stack, the decisions should always be kind of “Okay, what will enable us to kinda stay focused on creating value, versus kinda being maybe, in some ways, slaves to the toolchain?”

    Shimel: Got it. All right. So let’s jump in a little bit – you know, we got off on a bit of a tangent, but let’s talk about Semaphore.

    Anastasov: Sure.

    Shimel: What is it? How did that – I mean, I think you’ve given us a little background on how it came about. Talk to us what’s Semaphore about.

    Anastasov: So Semaphore is – generally, our mission is to enable teams to build great products at high velocity. And we do that by making it as easy as possible to enable continuous integration and continuous delivery processes in their organization. We’ve started way back in – about six years ago, when we launched Semaphore initially, it was like we were looking at the tools available and, from our perspective, services like GitHub or, you know – on GitHub, you do a few clicks and you get your repository; you’re ready to collaborate with other people. We looked at, for example, there were already platforms-as-a-service, like Heroku, for example, where, again, in a couple of commands, you kinda have your entire application up and running and you’re ready to kind of iterate and get something to customers very quickly.

    Shimel: Mm-hmm.

    Anastasov: But, in the area of, for example, just having a scalable continuous integration process, basically, at that point, Jenkins was the only tool that’s available. And kind of this was initially like “scratch your own itch” kind of thing, but, for a lot of people, maintaining and scaling and learning how to do all that, the infrastructure for building and testing their code isn’t really their core business purpose or where they kind of create value. So our initial goal was like, “Okay, can we create something which is as easy to use as, for example, GitHub, but for continuous integration?” so that you just kinda click to connect your GitHub account, select your repository, and we, in many cases, automatically infer what the CI build should be and kind of you’re just off the ground. You have the full continuous integration process automated and running in the cloud.

    And so, pretty soon after that, of course, listening to our customers, we implemented some features that enable continuous deployment so that customers or open-source users can keep track of releases and manage different environments. And so what’ve – throughout the years, we’ve kind of evolved from – so, traditionally, it was mostly about when people think about this space as “hosted CI,” so it’s kinda like there’s some – but, at this point, there’re a couple of options, of course, other companies doing the same function. And I think the perspective’s like, “Okay, it’s very – ” but, at this point, 2018, it’s pretty straightforward to kind of automate the testing, using some kind of a cloud service.

    But that’s not – if we go back to kind of what we also discussed in the beginning, if the goal is to accelerate kinda the entire engineering organization, there’s more to just running tests, you know? So every tech leader wants to – I mean, their objective is to deliver bug-free products to customers as quickly as possible. So we’re talking not just about testing but also configuration and delivery processes that often involve multiple technology stacks, which, of course, evolve over time. Now we have container technology, for example, which we didn’t have –

    Shimel: Changed the game.

    Anastasov: Completely changed the game. So the way to do this is to basically implement continuous delivery so that, step one, you automate all the processes and step two is to kind of optimize them, but using Semaphore for automating all continuous delivery processes has so far been – it wasn’t able to support, let’s say, every process that’s needed, so this is why, this November, we’re excited to present the brand new version of Semaphore, which is —

    Shimel: That’s Semaphore 2.0, right?

    Anastasov: Exactly. Yeah.

    Shimel: Mm-hmm. So, Marko, let me ask you this. I think you guys are pretty proud; on your website, you mention your bootstrap. You know, it’s interesting and, when you look at the CI/CD space, there’s a lot of great open-source, free tools, right? Jenkins, obviously, is the obvious one, but CloudBees, of course, has the commercial version, if you wanna call it. CircleCI, Travis – what’s the other one? – City – I forget the name of it. You and I both know there’s several pretty popular CI/CD tools, ARAs and all of this stuff. What do you think the special sauce is for Semaphore, if we’re talking – and our audience is developer-heavy, DevOps-heavy – why Semaphore versus maybe some of these others?

    Anastasov: Sure. We’ve always been focused a lot on speed, so “How can we not just provide a tool that automates something for people, but can we give them something preferably out-of-the-box, so that they just save time, comparing to whatever else they were using previously?” So one of the kind of foundational things, when you use Semaphore, is that basically we kind of periodically keep measuring this and it always stays true, is that Semaphore is twice as fast, comparing to all other competitors in the cloud space, in terms of just pure performance.

    So this alone can kinda save hours of engineering time to any team. Of course, it depends on the size of the projects and the team, but, if you kinda think back, kind of the fully-service hourly rate of developer in US is about $150.00. So, if you can do something to optimize that, we think it’s a huge win. Over time, we’ve kind of made steps forward, in terms of –

    [Phone ringing]

    Is this somebody from Jenkins calling to – [chuckles]

    Shimel: Yeah, it could be. [Laughs] That was pretty timely, but, no, I think it was something else. Go ahead.

    Anastasov: Yeah. So we provide tools that, for example, automatically paralyze the CI builds for projects, so people – which is, you know, large projects, for medium or bigger-sized teams with 100 or more developers, they typically have CI builds that run for hours. And the feedback loop – the kind of feedback loop in which you find yourself working, as a developer in such a project, it’s a pretty miserable experience. So you kind of become defensive. You work in wandering branches; you don’t merge off and – you know, you have flaky tests and so on. So, essentially, what we’ve seen, when people adopt Semaphore is, if they start using our automatic paralyzation feature, they go from, in some cases, from over 2 hours to something like 15-minute CI build. And it’s literally like they just needed to drag a slider and it worked. So we try to do things that really kind of improve the productivity of the teams.

    And so far, on the market also, it hasn’t been – in Semaphore 2.0, we’ve kinda took a lot of kinda learning that we picked up along the way and from our customers, especially the bigger customers, and, first of all, we made it really easy to create complex pipelines for companies that need to model the continuous delivery processes. On the other hand, what we think is pretty important is that, so far, this kind of cloud-based CI and CD tools – and Semaphore was part of this. In fact, we kind of initiated this idea of kind of charging per worker or per concurrent some kind of a machine that’s executing your stuff, your jobs.

    So, with Semaphore 2.0, we made the decision to kind of provide a service which is, let’s say, truly cloud, so it scales automatically to your needs, so you only pay what you use and you never really have any waiting time, which is a huge bottleneck we see in many organizations. They would maybe purchase some resources on a monthly or annual basis from Semaphore or Circle or some other product, and it’s like, if they are, let’s say, really busy in a certain period of a day or week, their CI just doesn’t process what they need. So we think that, with auto-scaling, developers will always be able to kind of get their work done quickly and stay in the zone as much as possible.

    Shimel: Got it. Hey, Marko, man, I would love to talk to you for another hour about maybe state of developers. And I do think you’re right – while speed is really important, as a benefit or an advantage to Semaphore, I think we’re in a world where developers are the top alpha predator, right? The most difficult kind of shoes to fill, the hardest jobs to get. Giving them an environment that keeps them happy, that keeps them working, if you will, in a good way, is a huge bonus than, frankly, making them work with crappy tools. I’m not calling anybody’s tool “crappy,” but I’m just saying, given the choice, I think developers would – they’d wanna work with tools that make their life easier and make it a better experience for them. But we’re at a half-hour and this was way more than 15 minutes, man, so we’re gonna need to end this. Maybe we can pick it up a second – in another chat in a couple weeks or whatever.

    Anastasov: I’d love to.

    Shimel: But, for now, we’ll call a wrap on this episode of DevOps Chat. Interesting conversation with Marko Anastasov. Close?

    Anastasov: That’s correct. Yeah.

    Shimel: All right. Cofounder of Semaphore. The new Semaphore 2.0’s coming out; check it out – great tool for CI. Until next time, this is Alan Shimel for devopsy.kinsta.cloud and you’ve just listened to another DevOps Chat.

    — Alan Shimel

  • Electric Cloud Extends Continuous Delivery Platform

    Electric Cloud Extends Continuous Delivery Platform

    Electric Cloud is making available a software-as-a-service (SaaS) pricing option available for its ElectricFlow application release management and continuous delivery (CD) platform.

    In addition, version 8.5 of ElectricFlow adds support for a Kanban-style pipeline view, object tagging for custom reporting and improved searchability and a continuous integration (CI) dashboard to track and analyze build processes, failures and successes.

    The SaaS option doesn’t fundamentally change the existing Electric Cloud pricing model as much as it does provide a different location to host ElectricFlow, said Electric Cloud CTO Anders Wallgren. Pricing for ElectricFlow is still based on nodes and the number of users rather than consumption.

    The CI dashboard, meanwhile, provides organizations that have one or more types of CI platforms being employed with visibility into, for example, Jenkins, Git and Bamboo pipelines using a single dashboard. Many organizations today have multiple CI pipelines in place largely because of the preferences of individual developer teams, Wallgren noted.

    The Kanban-style pipeline also provides a mechanism for managing multiple application development projects based on a just-in-time process originally developed by Toyota that is employed to identify bottlenecks in car manufacturing. That same issue, of course, applies to software development, where the number of dependencies between various modules is increased exponentially. That issue is becoming even more pronounced as organizations move to build and deploy cloud-native applications based on various types of runtimes. DevOps teams now can approve and observe release progress and deployment status within ElectricFlow in near real time, Wallgren said.

    Finally, tagging functionality on all objects allows release managers and product owners to create more granular, real-world reports quickly and easily. Universal data segmentation replaces code-based report creation and can be applied across any object type.

    Wallgren noted that adoption of DevOps best practices generally continues to advance at uneven rates. Electric Cloud is now working closely with the DevOps Institute on an online training and certification program dubbed Electric Cloud University. The goal is to increase the number of IT teams that are exposed to modern CD processes at whatever pace they choose. Electric Cloud is making four certifications available: Build Accelerator, Test Accelerator, Administrator and Agile Master.

    Wallgren said the next major frontier when it comes to continuous delivery will be applying deep- and machine-learning algorithms to tame what has become an exceedingly complex management challenge. DevOps toolchains continuously create vast amounts of data originating from numerous tools, and machine learning algorithms can harness that data to automate a range of DevOps processes in a way that is not only makes those processes more accessible, but also easier to demonstrate how an application release meets a specific compliance mandate. To achieve that goal, Electric Cloud has been embedding more algorithms into every release cycle of ElectricFlow, says Wallgren.

    It may take a while longer for algorithms to minimize all the complexity associated with continuous delivery today. But as DevOps continues to evolve, it’s apparent that many of the manual tasks that have often conspired to limit adoption of DevOps will one day soon disappear.

    — Mike Vizard

  • DevOps: Build a Continuous Improvement Culture

    DevOps: Build a Continuous Improvement Culture

    DevOps was built around the concept of continuous everything. We even use it in many of the names—continuous delivery (CD), continuous testing, continuous integration (CI)—and it is a worthy goal to want to improve the places that form bottlenecks in your software delivery systems and processes.

    But what about the bottlenecks in the DevOps systems and processes?

    Like any other system, DevOps involves tools. We choose those tools, invest in learning those tools and use them to do great things. But then we have investment in the tool. If our knowledge and systems for deployment are all based around CD tool X or ARA tool Y, we are just as unlikely to want to move from it as we were unlikely to want to move away from the previous toolset.

    Even if you discount the number of shops that have done pilot or small-team DevOps but kept the core of their business waterfalling, there are still a large number of organizations that stepped into DevOps and stopped improving. Granted, it is a huge job to move a significantly sized application portfolio into the DevOps mold and toolchain, but it is contrary to the process/goals to stop just because you did.

    The market is nowhere near finished evolving: New tools are coming available and massive improvements to existing tools are being released. Keep taking advantage of those facts.

    The more projects/portfolios you have moved into DevOps, the more weight there is to resist change. That is the simple reality, just as the larger a rock, the harder it is to move (assuming the same type of rock, of course). The answer: Watch for those improvements that will make moving the rock worth the force required to do so.

    Working containers into the overall DevOps process, for example, can benefit every step in the toolchain. But if they’re not already in the process, it is a lot of work to get them folded in. I chose this example because for most shops, a little research will show it is well worth the effort. From preconfigured Dev/Test/Prod servers to run-and-done compile instances, containers can make the process more manageable and more agile. The benefits outweigh the costs for most shops.

    So keep an eye out. Don’t implement and spend all of your time on the tools you’ve chosen. Stay aware of what is out there, and what is coming. Understand how it would impact the DevOps process and try to pull in those items that fit the environment.

    And, as always, keep rocking it. The end goal is to keep the systems up, running and usable. And most of you are doing a great job.

    (This, and all blogs, brought to you as much by the geeks behind the scenes as the author and editor.)

    — Don Macvittie

  • Standardization, De Facto Style

    Standardization, De Facto Style

    The first decade or so of DevOps had one truth to it: It was about change. Not just change in the way that IT worked, but change in the way DevOps worked—particularly in the tools space, where products and product preferences both are in flux. We’re still seeing a massive change, but increasingly the effect is more like the ripples created when a rock is thrown into a pond than a tsunami, dumping change upon change.

    You could look at this one of two ways (and no doubt those who have more reason to dig into details than I have already done so): Either the slow but steady standardization typified by tools such as Jenkins and Puppet have enabled participation of enterprises in the DevOps life cycle, or the demands of enterprises for some amount of stability in their massively complex systems have driven standardization.

    Either way, the result is the same. We are seeing a lot of change further out, but the core technologies for build/test/deploy are settling into a few products by a few vendors. That is good for adoption, even if you would argue that it is bad for innovation. It means that skill sets are, at least to a certain extent, portable across DevOps teams and organizations, which for a long while was not true. When I was knee-deep in deployment technology, knowing how to use product A did not mean you knew how to use product B. The more people using one or the other, the better for employee mobility and the pool of available resources, as both people and support tools like standardized scripts.

    I recently was going through the offerings of a service provider in a space that doesn’t traditionally have DevOps enablement. The company’s offering included DevOps, so I drilled down. Seeing the company supported Jenkins, Jira and Xebia Labs made me feel better about the program it was putting together. It meant the company was adapting mainstream tools to a specialty market.

    That’s what spurred this blog. It’s been a long time that these things were seen as “new and shiny”; now, for a service provider to include them as proof it is serious about customers’ problems shows just how far we have come.

    There’s still a long way to go. The DevSecOps community is currently in the throes of growth, and lots of outer-edge DevOps tools are incompatible with each other and chosen per shop without a clear market leader. But we’re getting there. And at this point, a person going into DevOps who asks me what they should know will actually get an answer of “Learn X, Y, Z” rather than “You should learn X, but pick up some of X.1, X.2, and A too, then there’s Y …”

    The thing is, though de facto standardization is happening, to many (particularly enterprise) shops, this is all still new. So if you don’t know the recognizable tools yet, don’t worry too much; there will be opportunity to learn as they are rolled out across a given org.

    — Don Macvittie

  • CloudBees Acquires Codeship to Extend CI/CD Reach

    CloudBees Acquires Codeship to Extend CI/CD Reach

    CloudBees today signaled its DevOps ambitions to extend well beyond the Jenkins continuous integration/continuous deployment (CI/CD) platform by acquiring Codeship, a provider of a CI/CD platform that is delivered as a software-as-a-service (SaaS) application.

    The decision to acquire Codeship is being driven by the need to engage smaller DevOps teams that don’t have the time or expertise required to set up on-premises instances of Jenkins, said CloudBees CEO Sacha Labourey.

    Codeship today is being used by more than 100,000 developers spanning 2,400 customers. Many of those developers are working at smaller companies. But Labourey noted that small teams of developers inside larger companies also prefer to invoke a CI/CD platform as a service.

    “Different teams will decide to do different things in the same organization,” said Labourey.

    The Codeship platform is not based on Jenkins and there is no plan to rehost the Codeship service on Jenkins, says Laborey. Jenkins is designed to be highly customizable by the organization that employs it. The Codeship service is a highly opinionated instance of multitenant CI/CD platform that is designed to provide a turnkey experience capable of support lots of developers at once, explained Labourey.

    CloudBees will explore ways to bundle its core platform with the Codeship service for organizations that find themselves needing to deploy both a CI/CD platform on-premises in addition to a cloud service, he added.

    The acquisition of Codeship signals a wave of coming consolidation across the DevOps sector, Labourey said, predicting that over time, a much smaller number of companies will account for most of the DevOps software and services sold. Some will rise from the ranks of the pioneers of DevOps. Others will be the same companies that today dominate application lifecycle management (ALM) and software development lifecycle (SDLC) software in the enterprise. As DevOps continues to gain momentum in the enterprise, it’s now only a matter of time before legacy ALM and SDLC vendors are forced to modernize their portfolios by making acquisitions or partnering with companies such as CloudBees, he said.

    It’s too early to speculate how that consolidation might play out. But it’s clear the rise of microservices is about to force the DevOps issue in many enterprises. The rate at which DevOps processes have been adopted across enterprise has been uneven, but as organizations embrace microservices to build better applications faster, many will need to re-engineer their application development processes. There always will be monolithic enterprise applications that may be better-served by waterfall approaches to application development, but in time, the bulk of applications will be built employing microservices architectures that make it easier to update and manage applications employing modern DevOps best practices.

    How much those DevOps processes will be based on CI/CD platforms running in the cloud or on-premises is anybody’s guess. More than likely, most organizations will rely on a combination of both, depending on the size and scope of their efforts. Regardless of the approach, however, at a time when every company is becoming more aware of their dependency on software to compete, it will be those organizations that embrace DevOps processes best that will prosper at the expense of rivals.

    — Mike Vizard