Category: Continuous Delivery

Continuous delivery is a software engineering discipline that allows organizations to consistently develop and release quality software at a rapid pace. It is achieved by automating the software development lifecycle, enabling tasks such as development, testing and release to occur concurrently and continuously.

  • 7 DevOps Lessons in 2018: The Path to DevOps Success in 2019

    7 DevOps Lessons in 2018: The Path to DevOps Success in 2019

    We all learn from mistakes, right? But it would be better if we learned faster and didn’t repeat them in our future exercises. And this is exactly what we will discuss here in this blog.

    For a while now, we’ve been overwhelmed by the idea of the DevOps principle. We love how the collaboration between development and operations provides room for automation, continuous integration/delivery and enhanced time-to-market.

    But what about the tiny shortcomings we overlook in the process of continuous delivery?

    Listed below are some of the don’ts—or, rather, lessons learned from our DevOps implementation in 2018:

    Gauging Employee Capacity Over DevOps Delivery

    When Carmen DeArdo, Tasktop’s senior value stream management strategist, posed two questions to a crowd of 200 people, they had no answer to give. What were the questions, you might ask?

    • How many treat their delivery pipeline tooling as an integrated product with a product owner and an architect?
    • How many have any measure of how long it takes for them to deliver a feature or defect?

    Companies treat their delivery pipeline tooling as different components along the delivery chain. And, when it comes to measuring time taken, all that matters is getting the product to the finish line.

    These components are important if companies need to avoid obvious mistakes on the path to success. Consider the above questions the next time you set out on your DevOps mission. Within this aspect, measuring team happiness is as important as an integrated delivery pipeline tooling, and time is taken to introduce a feature.

    Companies need to strike a balance between managing work and achieving team happiness. This is important if the company has a long-term vision. Unhappy employees burdened with work don’t deliver the expected results. A sustainable pace helps companies ensure that employees don’t succumb to pressure.

    Revise the Management Strategy

    Organizations need to encourage their managers in a way that they are not put off by the DevOps process but rather motivated to build and maintain their team.

    Most organizations emphasize giving managers incentives to deliver expected results, maintain quality, expand team size, set immediate goals and rake larger profits. However, this strategy could backfire when applied to a DevOps implementation.

    Companies instead should encourage their managers to create an environment where the teams grow and become flexible, not only contributing to that process but also adding value to other processes, hence, the organization’s overall success. This thought process will lead to a chain of positive outcomes.

    Upskilled and Cross-Skilled

    Gone are the days when specializing in a particular skill was rewarding! The competitive market is looking for a workforce that is upskilled and cross-skilled. In short, employees need to be a package of multiple skills, with an emphasis on soft skills.

    Being multi-skilled not only supports transformational goals but also supports an organization’s goals. When the lack of new skills becomes a constraint, companies must invest in hiring individuals to propel transformation, which is equivalent to investing in automation.

    Automation ≠ DevOps Success

    Somewhere between adopting the DevOps culture and being DevOps-equipped, professionals have shifted from the purpose of DevOps to automation.

    In the process, people have lost control over the overall management of DevOps. Either professionals are in a situation in which the code is not built in the intended way or that the automated part is managed but the steps required for automation are not.

    Organizations instead should refocus their attention on the scalable nature of DevOps, building a new, fully automated process rather than automating an end-to-end process over time, as it is difficult and also time-consuming. Hence, automation does not equal the DevOps mission accomplished!

    From Idea to Blackboard

    DevOps once was more an idea of a team discussion rather than full-fledged process. But we saw 2018 transform DevOps from a mere idea to a part and parcel of everyday business—so much that it defines the way businesses plan their finances, regulations and organizational strategy as a whole.

    Additionally, DevOps experts in 2018 emphasized the need for value delivery than just focusing on details of technical practices. This can be achieved by constructing a managerial framework that bridges the gaps between business and technology.

    Relieve Your DevOps of Old Tech

    Last year also witnessed DevOps professionals moving toward new technology such as containers, Kubernetes and serverless. Why? It’s obviousthese technologies enable DevOps to deliver its promises. Trying to innovate the DevOps process or even innovate its end product with old technology will only take you backwards in the DevOps process.

    In 2019, we should realize that holding on to old technology will mean limiting your organization’s capabilities and impact.

    Identifying the Right DevOps Adoption Patterns

    While there are several DevOps adoption patterns out there, some organizations choose a central DevOps team enabling standard CI/CD tooling. The approach is in sync with DevOps thought leaders’ approach of reducing global complexity.

    However, this approach fails at catering to the needs of different teams. With the focus only on addressing CI/CD tooling, many centralized teams struggle with adoption.

    In another approach, companies seek to engage ambitious and autonomous tools to use DevOps tools and practices.

    This approach tends to match the common DevOps belief, ‘First prove that new work methods fit the enterprise, then spread.’ However, these teams are only successful in delivering projects with high levels of automation involved but have quickly failed to convert team-level experience to company-level.

    A 2019 DevOps adoption approach should focus on having high levels of team autonomy and reducing global complexity.

    Here are some tips to help you achieve this:

    • The central architecture team collect and share recommendations about languages, libraries and tools.
    • Teams consider these recommendations and experiment to see if there’s an opportunity for greater gain.
    • If things aren’t working, teams can change change and undo dead-end technology decisions/ideas.

    Now that we are ready with our lessons, it’s time to go live! Are you ready for your DevOps 2019 showdown?

    — Veritis

  • 10 Years of DevOps—Transition, Growth and the Future of AIOps

    10 Years of DevOps—Transition, Growth and the Future of AIOps

    When software development became a cumbersome task of repetitive coding, testing and glitches, organizations had to come up with a solution.

    The solution? An organized approach that would not only eliminate the endless cycle of repetitive tasks but also would establish proper coordination and execution in the software development life cycle (SDLC).

    That very systematic approach with fewer glitches, more production and exemplary coordination today is called DevOps.

    But where did this wonderful process start? How did it make its pathway into our the cycle of IT industry?

    The Beginning

    It all began one day when Andrew Clay Shafer scheduled a session, “Agile Infrastructure,” at the Agile Conference in Toronto.

    However, the DevOps model back then didn’t seem to make much of a difference to those associated with software development. As a result, Shafer closed the session. But something struck developer Patrick Debois that day, who was presenting a paper on “Agile Infrastructure and Operations” in the same vicinity. Shafer’s theme seemed to draw his attention and he went straight to meet Shafer.

    A long talk in the hallway that day ignited the fire of DevOps, which started with Shafer and Patrick three months after their first meeting.

    Together, they started the Agile Systems Group, with an aim to foster greater collaboration between developers and operators.

    Today, the DevOps process continues to survive on the principles of increased collaboration and automation to build, test and release software faster.

    Mobile Device and DevOps Adoption

    Over the years, DevOps began to grow and pick up fame. And with the emergence of mobile phones, the trend picked up rampant speed.

    According to a market intelligence company App Annie, global economic activity concerning mobile applications will be worth USD $6.3 trillion by 2021, five times 2016’s revenue.

    Creating software for mobile devices requires an elevated software development approach. Now, organizations have to deliver new, high-quality software to customers as soon as possible to remain atop the competitive market.

    In this new approach, organizations need to focus on two prime factors: agility and adaptability. While the former defines the ability to code and recode, the other marks DevOps’ ability to include changes along the SDLC.

    Companies are constantly working on getting these qualities into their workflows to promote DevOps concepts of continuous delivery, continuous integration and eliminating bugs to allow users to enjoy a hassle-free software experience.

    DevOps Over the Past 18 Months

    According to Gartner, about two-thirds of organizations were using DevOps to support the certain business process by the end of 2017, which Forrester termed as the “Year of DevOps.”

    Forrester awarded the term to DevOps owing to increasing clients’ questions and discussions around, “How do I implement DevOps at scale?” rather than just, “What is DevOps?”

    Going by DevOps’ potential to support modern business processes, more and more companies are adopting DevOps, leading to a steady rise in the trend.

    As DevOps continues to be successful within and across the organization, the IT industry will continue to see what it looks like to be “DevOps successful.”

    The Emergence of DevSecOps

    While DevOps continued to change the software industry, however, security within the process still remained an unanswered question.

    In 2012, Gartner analysts Neil MacDonald and Cameron Haight introduced the concept of DevSecOps. This meant security’s introduction right from the beginning and continuing throughout SDLC.

    Security’s entry into the model made DevOps a more alluring practice. Six years out, DevSecOps has been a much-discussed concept across seminars, conferences and written content.

    But it is still a question whether organizations are implementing DevSecOps considering the time constraint, the need for accelerated delivery and the understanding of security’s importance in the cycle.

    For many organizations, security is still not a part of their production cycle. But, it is essential for any organization to consider security if they want to remain in the competitive market and importantly at the top.

    Negligence in this area may cost them bankruptcy or even elimination from the industry. According to the “Global State of Digital Trust Survey and Index,” one-third of British customers stopped using an organization’s services owing to a security breach.

    10 Years of DevOps: What’s Next?

    The emergence of AI and automation have made some waves in the IT industry, and DevOps is no exception.

    DevOps holds high predictions for the future. In the years to come, AI and automation will change the software development process. Apps will be self-written and coded. Tasks such as integration, security, testing and deploying will be dictated by AI, data and machine learning in the all-new AIOps.

    A data-driven approach will have software dynamically adapt its behavior on the basis of what is in its environment, much like other AI-driven technology including Apple’s Siri and Alexa.

    Moreover, this significant shift from programmed to non-programmed apps will call for improved security and functionality management and allow developers to focus on value-addition activity.

    DevOps is evolving. How about your organization?

    — Veritis

  • Role of DevOps in Mobile App Development

    Role of DevOps in Mobile App Development

    Emerged out of Agile methods in 2001, DevOps basically represents a fundamental departure from the traditional waterfall software development approach.

    DevOps is a modern software development method that focuses mainly on the effective collaboration between software developers, project managers and operations staff, while also aligning with business goals at the same time.

    DevOps came into being when developers and operations staff came together to more effectively overcome the challenges associated with continuous software delivery.

    DevOps in Mobile Applications

    Peerbits takes a holistic view of mobile DevOps, defining it as a capability for continuous mobile app delivery that enables our clients to reduce time-to-market new features and seize new market opportunities simultaneously.

    The key principle in DevOps is continuous delivery, which means deploying the software (in this case a mobile app) and the environment on which it runs, automatically and on-demand, during any stage of the mobile app development cycle.

    DevOps basically applies the same principles whether you want to build an app for either mobile or web. However, the challenges in mobile DevOps are bit different and specific.

    Mobile DevOps Challenges

    The challenges in mobile DevOps mainly include:

    Different OS Versions: Most mobile apps have multiple target environments and multiple devices, which require dealing with different specs, different OS versions and different form factors.

    Furthermore, not only is there a wide range of operating systems running in the market, but there are also different installed versions.

    Take Android, for instance: The Android market accounts for almost 85 percent of the smartphone market. The below mentioned chart shows the breakdown of installed Android Versions as of Oct. 26.

    Image source: www.developer.android.com

     

    As you can see from the chart, the Android version 6.0, which according to Wikipedia is now unsupported, is still being used by 21.3 percent of Android users.

    This clearly indicates that only a few users update as soon as the new version comes along, while others don’t, either because they don’t want to or because their smartphone manufacturers are not providing newer OS updates.

    In addition to this, most Android smartphone manufacturers also tweak newer Android OS slightly, resulting into an increase in the OS fragmentation and making the launch times slower.

    Hardware: Smartphone hardware presents two primary challenges. First, smartphone manufacturers offer a good range of chipset, memory, storage options and, most importantly, different screen sizes. This makes it difficult to support each of them.

    Second, the mobile device war no longer is being fought only between Apple and Samsung. There are new smartphone manufacturers entering market more often than before. Out of these new rivals, there are also many manufacturers that offer low-cost models.

    Though Apple and Samsung might be dominating the Western markets right now, in China both of these brands account for only 13.7 percent and 20.7 percent respectively.

    In 2015, at least 18 new smartphone manufacturers entered in the market in Asia.

    This localization and fragmentation of available devices and their different OS versions adds a considerable complexity to the mobile application development as a challenge.

    Balancing Mobile App Quality with Always Changing Consumer Demand: As the app economy grew in last recent years, the need for businesses to get their mobile apps released in the market quickly also grew along with it.

    However, since the consumer expectations are also increasing, the quality of a mobile app is the most essential factor in retaining existing customers as well as attracting the new ones.

    In 2018 especially, this is more and more difficult, due to the ease in which customers can delete apps and give poor ratings and review to low-quality mobile apps.

    As a result, businesses need to be extra careful in terms of their app quality, while consistently adding value to keep their existing customers happy.

    All of these factors are putting strain on mobile app development teams, as the waterfall model approach is not a sustainable practice for today’s consumers.

    3 Rules for Successfully Adopting Mobile DevOps

    Adopting DevOps in your business mobile app needs only three considerations, which are explained below.

    Continuous Planning and Integration

    Continuous planning refers to bringing your entire mobile app project team—developers, project managers, operations staff, and other stakeholders—on the same platform to finalize the scope of the mobile app in the preparation of a launch plan.

    Continuous integration, on the other hand, ensures that the code submitted by one developer works with the code submitted by the other developer(s). Simply put, continuous integration in mobile DevOps demands frequent builds, which must be integrated with the last developed code.

    Continuous Testing and Monitoring

    Most mobile application testing is executed on simulators instead of actual devices. Plus, the testing is also manual. And given the number of OS versions alone, it is impossible to do manual testing for all versions.

    A mobile app can perform well in test environment, but it is very common that it will fail in the live environment.

    The reasons of these failures can be many, such as memory, network conditions, power, etc. It is necessary, therefore, for developers as well as businesses to make sure that the continuous monitoring is done via third-party SDKs for log, crash report and so on to find the root of the problem.

    Continuous Delivery and Deployment

    Continuous delivery refers to the practice that makes sure the code is deployed to the production environment by submitting each change to the production-like environment.

    Continuous deployment is the next step of continuous delivery. In continuous deployment, each change that has been approved by continuous testing gets deployed to the production environment automatically.

    Takeaway

    In our perception, there is no such thing as a separate DevOps for mobile application development. DevOps is a general approach that works for all components as well different kinds of application development.

    While there are certain challenges involved in adopting DevOps for mobile apps, but there also measurable benefits that makes the entire journey worthwhile.

    All in all, despite the cost to build an app and its initial pains, the benefits of DevOps make the investment worthwhile.

    — nasrullahpatel

  • CircleCI Automates Configurations Across CI/CD Platform

    CircleCI Automates Configurations Across CI/CD Platform

    CircleCI has unveiled what it describes as the first package manager designed specifically to address configuration tasks involving software delivery automation.

    Matt Wyman, vice president of product for CircleCI, said the Orbs package manager condenses commands, executors and jobs into single, reusable lines of code that automate tasks on the continuous integration/continuous delivery (CI/CD) platform the company provides as a cloud service. The initial set of Orbs being made available by CircleCI are focused on automating more than 25 different cloud deployment scenarios to enable organizations to employ DevOps processes at scale, he said, noting those Orbs can then be validated and shared via an Orbs registry that CircleCI is making available.

    CircleCI is also launching a technology alliance program through which Jfrog, Twistlock, Anchore, Aqua Security, Cypress, Postman, Coveralls, datree, Rollbar and Codacy have committed to building Orbs.

    Wyman said shareable Orbs that require only a few lines of code to develop makes it possible to share configurations across multiple CI/CD workflows. Those Orbs then can be integrated with any number of third-party tools.

    CircleCI is making a case for a CI/CD platform delivered as a managed cloud service that eliminates the need for DevOps teams to set up and manage on their own. Open source CI/CD platforms can be too complex for the average IT organization to set up. Alternatively, CircleCI makes available a version of its CI/CD platform that organizations can deploy behind their own firewall. The CircleCI approach is designed to enable DevOps teams to flexibly create workflows that can be executed sequentially or in parallel.

    There are 12 million builds a month being generated by more than 300,000 developers taking advantage of the CircleCI cloud service to build applications, Wyman said. One of the primary drivers of that success is a cloud service that allows organizations to apply more of their limited resources to developing applications versus managing CI/CD infrastructure, he noted.

    Most IT organizations have yet to standardize on a specific CI/CD platform. Competition between providers of CI/CD platforms has never been more fierce as more enterprise IT organizations embrace DevOps. As DevOps processes continue to mature, it’s also clear that DevOps processes will need to be extended across multiple clouds in addition to on-premises IT environments.

    In general, CI/CD platforms are initially set up by developers who have embraced DevOps processes to accelerate development and deployment of applications without sacrificing quality. The most widely used CI/CD platform is the open source instance of Jenkins. Developers often can adopt Jenkins on their own without requiring anyone to sign off on a purchase order. But as organizations move to scale up their DevOps processes, CircleCI is betting interest in a commercial service that can be easily accessed across an entire enterprise will increase as organizations move to standardize their CI/CD environments.

    In the meantime, pressure to increase the rate at which applications are built and deployed is only going to increase as organizations become more aggressive about digital business process transformation. The issue now isn’t so much whether they will have a CI/CD platform, but rather to what extent—and, in some cases, how many.

    — Mike Vizard

  • Semaphore 2.0 Brings Continuous Delivery to Teams of All Sizes

    Semaphore 2.0 Brings Continuous Delivery to Teams of All Sizes

    Since its inception in 2012, Semaphore has been focused on making code testing and deployment a quick and easy process. The cloud-based CI/CD platform from Rendered Text enables users to get up and running within a matter of minutes—which the company claims is at least twice as fast as other offerings.

    The company is releasing Semaphore 2.0 Nov. 6, which will expand the platform to enable continuous delivery, for a product that allows teams of any size to automate any continuous integration or continuous delivery workflow with custom pipelines so users can model their CI/CD however they prefer. The latest version also better supports CI/CD for containers, giving users the ability to use Docker or Kubernetes tools in building within container environments. Other features of Semaphore 2.0 include support for YAML files and command line interfaces, so teams can manage a large number projects more easily.

    Marko Anastasov, founding partner at Rendered Text, spoke with me about the latest version of Semaphore and his thoughts on the continuous integration and continuous delivery space at large. Read on for his insights:

    Can we start with some background about Semaphore? What was the catalyst for creating it?

    Our company was an early adopter of agile software development practices, including test-driven development and continuous deployment. We relied on cloud-based products like GitHub, which were simple to use and made software development easier.

    However, when it came to implementing continuous integration, we were out of luck. I remember spending days setting up Jenkins for just one of our projects. We saw how expensive and time-consuming it would be to make it fast and scalable. It felt as if by continuously verifying our code, we were doing something wrong. Developers need to set up a fast and functional CI process in minutes, and not worry about how it works.

    What’s new in Version 2.0 that’s worth mentioning?

    Semaphore 2.0 is made to enable continuous delivery in organizations of any size. Previously, like other hosted CI/CD services, Semaphore focused on solving the integration process: building code and running tests. Delivery features were limited and tailored for classic web development.

    This time we asked ourselves, What would a continuous delivery product focused on speed and simplicity look like in the age of cloud-native computing? I’d highlight three things that stood out.

    First, for DevOps transformation to succeed, we need to automate, visualize and accelerate all software development processes in our organizations. So at the heart of Semaphore 2.0 are infinitely customizable code pipelines, configured as code, run as a service.

    Second, infrastructure should be ephemeral and scale in real time. When CI/CD capacity is limited, developers are blocked in busy times of the day. Semaphore adopts the “pay only what you use” cloud model, in which CI/CD resources scale automatically to support the team’s actual needs.

    And third, we can’t manage what we don’t measure. It doesn’t make sense to prescribe the same metrics for every project and organization. So Semaphore lets users choose their own metrics and define their own dashboards.

    What feature or features do you believe will have the most impact with users?

    In building Semaphore 2.0 we were very careful not to add any unnecessary features. All parts are designed to support each other and provide a harmonious user experience.

    For example, Kubernetes wasn’t the first solution to run software in the cloud, but it made vendor-neutral cloud computing as easy as describing what you need to run. Similarly, developers can now describe any CI/CD process they need, and Semaphore will run it immediately at any scale.

    In day-to-day use, I expect users to like how much control they have via command line and autoscaling resources. It’s the overall boost in productivity that will make the most impact.

    How important is Semaphore’s increased support for container environments?

    Containers are the biggest change in the development stack in recent history. In the transition to container-based software delivery, many teams needed to maintain their own CI/CD infrastructure, since none of the existing solutions were flexible or easy enough to use. This takes valuable time away from delivering value to customers. Semaphore 2.0 is built to support the next-generation of cloud architectures based on containers.

    What are some of your thoughts on the current state of the CI/CD market, and where do you see it heading?

    For the continuous integration part of the equation, there are many options to choose from. There’s a wide spectrum from managed to DIY solutions, and from specialized vendors to those who provide basic tools for every aspect of software development. I think the shift that’s going to happen is from basic automation to optimization. For example, Semaphore provides bare-metal performance at convenience of cloud. When a customer tells us that they’ve combined that with our test optimization features to reduce build time from hours to minutes, that makes us so happy because we know what tremendous difference that makes in their workplace.

    On the continuous delivery front, I think there’s less maturity and that the market is still developing. So far, developers have needed to build their own solutions based on complex open source tools and self-managed infrastructure. Cloud vendors have entered the space, not with complete products but low-level tools that leave a lot of complexity for the end user to manage. The next phase will be about making the entire CI/CD process easier and more accessible. At Semaphore, we’re committed to making that happen.

    — Charlene O’Hanlon

  • DevOps Chat: CI/CD with the Semaphore Platform

    DevOps Chat: CI/CD with the Semaphore Platform

    As the old saying goes, “speed kills.” When it comes to development, a lack of speed can kill, too. Marko Anastasov and the Semaphore team have strived to make Semaphore the fastest continuous integration tool in the market. With version 2.0 of Semaphore, they believe it truly is.

    With his background in development, Marko brings a unique perspective in leading the Semaphore team. He knows what he and his colleagues like, and that is the road map for Semaphore. Have a listen to Marko tell us in his own words.

    As usual, the streaming audio is immediately below, followed by the transcript of our conversation.

    Transcript

    Alan Shimel: Hey, everyone, it’s Alan Shimel, devopsy.kinsta.cloud, and you’re listening to another DevOps Chat. In today’s DevOps Chat, we’re gonna talk about a company called Semaphore. Their website is semaphoreci.com. And I’m happy to be joined today by one of their cofounders, Marko – and I’m not gonna pronounce his last name well, so I’m gonna let him pronounce it himself. Marko, your last name?

    Marko Anastasov: Anastasov.

    Shimel: Anastasov. Correct?

    Anastasov: Yes, that’s correct. Hi, Alan, and thanks for having me. It’s a pleasure.

    Shimel: Thank you for joining us. So, Marko, before we jump into Semaphore, let’s give our listeners a little bit of background on your – who is Marko Anastasov? How did you get here today?

    Anastasov: Yeah. Yeah. Sure. So my core is, you know, I’m a programmer. Big fan of making – I’m a product-guy-slash-programmer. Maybe that’s kind of the intersection where I’m at, so I like making software products. Been doing that kind of almost since I was a kid.

    Shimel: Mm-hmm.

    Anastasov: And the kind of when I started being, let’s say, a professional is when things were going fast on rails, like Ruby on Rails, you know, Web development. And so I got into that, despite being kind of C++ nerd myself, originally.

    Shimel: All right.

    Anastasov: And I just kinda fell in love with ease at which you can kind of make something, put it online, create something useful for other people. I did a couple of kinda noncommercial projects and started a small consultancy, with my cofounder Darko, who is also a cofounder of Semaphore now. And, you know, we’re both also big fans of clean code, doing things – you know, software craftsmanship, all that kind of stuff. And, as we kind of grew ourselves professionally in that whole area, it wasn’t long till we, of course, realized the benefits of processes like, you know – it wasn’t called, really, “DevOps” at that time, ten years ago or something, but we identified, “Okay, continuous integration is obviously a process that any kind of – if you wanna keep moving fast, as you kinda work on a project, this is what you gotta do.”

    And so, through kind of adoption of these practices in our own company and kinda having an eye on an ambition to create a tool that other kind of developers will also find useful, we started prototyping and eventually launched Semaphore in 2012. And, from that point, it’s kind of all we do and it’s been a great ride so far, you know, leading all the way to today’s interview with you.

    Shimel: Sure. Hey, man, thank you. You know what? It’s funny; it’s not easy, but, when you ask people to tell them about themselves, you don’t wanna blow your own horn too much, but you wanna tell people what’s going. And I think you did a good job with it. So let me – before we jump into Semaphore, you mentioned something that I’ve heard from a lot of other developers, people in the DevOps space, whether they be vendors, practitioners, what have you, and that is they were doing what’s called a “DevOps-like things” before we called it “DevOps.” You know what I mean? And so were you into Agile? Was that something that kinda went with that?

    Anastasov: Yeah, I would say yes. Although I never – it was always – Agile itself has always been a bit of a vague concept for me. The first maybe introduction to that kind of thought space, for me, was the “Extreme Programming Explained,” the book by Kent Beck, which –

    Shimel: Sure.

    Anastasov: So it was more like a practitioner’s path for me, so, as you’re in development, working in small iterations, you know, you’re breaking things down, being ready to adapt to change, kind of not being married to some kind of roadmap, and so on. So it was definitely something that we identified as a good way to operate, but even before maybe the Agile backslash – I don’t know. I didn’t use the word “Agile” myself that much, but –

    Shimel: Well, you weren’t doing Scrum or taking Scrum classes and stuff.

    Anastasov: Not really.

    Shimel: But, certainly, sort of the Agile way of doing software development, right, before we called it “DevOps” or anything else, was kinda ingrained in you, if you will. So then I always – and this is something, Marko, I’ve asked other people about – so one of the unique things about DevOps is we sort of build on that Agile, we build on Lean, but, certainly, ops gets involved.

    Anastasov: Yeah.

    Shimel: When did you kinda make that jump, as sort of a programmer/developer, to say, “Hey, it’s really important we get the ops folks, the ops functionality, kinda built in there”?

    Anastasov: So –

    Shimel: And I don’t wanna put words in your mouth.

    Anastasov: Yeah, sure. So, for me, it was about – so I come from a kinda small company environment and that was kinda my beginning. And so, in that kind of setup, if you’re kind of being responsible for a product, means that you’re also responsible for the operational stuff or that there is maybe a shared responsibility among developers to some degree, but there’s always somebody who kinda specializes, at least for a specific project, for doing the operations. And there is – I think it’s very healthy, kind of, for a programmer to kinda be exposed to these things because you – as you kind of experience this firsthand, you kind of  – let’s say you’re ready to – you’re not maybe – you don’t have patience that much for things that kinda get in your way of getting the job done.

    So, if you kind of practice not repeating yourself in writing code, you wanna kind of practice the – try to practice the same thing, when you kinda deal with infrastructure. So I think that’s kind of the line of thought that brought us the kind of infrastructure-as-code, you know, kind of EDU and the movement and the tools force.

    And, once you maybe grow out of that kinda maybe very small team kinda environment, I think what you see then is you see maybe the bigger picture of, if you think of continuous delivery as a broader objective of being able to deploy anything that you’re working on at any point in time, as the ideal state that you wanna be in, then being really good at ops means that, for example, if you’re doing microservices, you’re not really that even Agile in the ’90s, maybe, way, if it takes two weeks for somebody to provision you the resources so you can just get started and write the first lines of code in your microservice. So if that can be reduced to five minutes or an hour or something, that’s really great, so.

    Shimel: Yeah.

    Anastasov: Yeah.

    Shimel: So this is something, Marko, again, back when I first started devopsy.kinsta.cloud, four, five years ago, there was this big debate over “Is DevOps different at a small company versus at an enterprise?” And, of course, there’s differences between working at a small company versus an enterprise, right? In small companies, you do wear multiple hats. You don’t have the luxury of having a full ops team and a full separate dev team and QA and business architects and all that kind of – you know, you don’t have that. Everyone pitches in and does what they gotta do, where, in large enterprises, you sometimes do have that stuff.

    And so, obviously, there’s a difference between them, but, at the end of the day, DevOps is DevOps, right? It’s about doing continuous integration, continuous delivery, and so forth. So I think that’s a – you know, what you’ve described is very not out of the ordinary. I think that’s sort of the path, the track that these things follow.

    Anastasov: eah. Yeah. I think it’s about – whether you’re coming from a small or you’re working at a big organization, ultimately, it’s all about “How do we move fast so that we just kinda of – ” the ideal state is when we’re all kind of in creation mode, all the time or as much as possible.

    Shimel: Sure.

    Anastasov: And it can happen in a small team as much as, I think, in a large team, that things can get stuck, if you maybe – I don’t know – pick the toolset that’s maybe not fit for your needs, where you kind of want to reinvent the wheel or – so I think that, regardless of maybe the team size or the technology stack, the decisions should always be kind of “Okay, what will enable us to kinda stay focused on creating value, versus kinda being maybe, in some ways, slaves to the toolchain?”

    Shimel: Got it. All right. So let’s jump in a little bit – you know, we got off on a bit of a tangent, but let’s talk about Semaphore.

    Anastasov: Sure.

    Shimel: What is it? How did that – I mean, I think you’ve given us a little background on how it came about. Talk to us what’s Semaphore about.

    Anastasov: So Semaphore is – generally, our mission is to enable teams to build great products at high velocity. And we do that by making it as easy as possible to enable continuous integration and continuous delivery processes in their organization. We’ve started way back in – about six years ago, when we launched Semaphore initially, it was like we were looking at the tools available and, from our perspective, services like GitHub or, you know – on GitHub, you do a few clicks and you get your repository; you’re ready to collaborate with other people. We looked at, for example, there were already platforms-as-a-service, like Heroku, for example, where, again, in a couple of commands, you kinda have your entire application up and running and you’re ready to kind of iterate and get something to customers very quickly.

    Shimel: Mm-hmm.

    Anastasov: But, in the area of, for example, just having a scalable continuous integration process, basically, at that point, Jenkins was the only tool that’s available. And kind of this was initially like “scratch your own itch” kind of thing, but, for a lot of people, maintaining and scaling and learning how to do all that, the infrastructure for building and testing their code isn’t really their core business purpose or where they kind of create value. So our initial goal was like, “Okay, can we create something which is as easy to use as, for example, GitHub, but for continuous integration?” so that you just kinda click to connect your GitHub account, select your repository, and we, in many cases, automatically infer what the CI build should be and kind of you’re just off the ground. You have the full continuous integration process automated and running in the cloud.

    And so, pretty soon after that, of course, listening to our customers, we implemented some features that enable continuous deployment so that customers or open-source users can keep track of releases and manage different environments. And so what’ve – throughout the years, we’ve kind of evolved from – so, traditionally, it was mostly about when people think about this space as “hosted CI,” so it’s kinda like there’s some – but, at this point, there’re a couple of options, of course, other companies doing the same function. And I think the perspective’s like, “Okay, it’s very – ” but, at this point, 2018, it’s pretty straightforward to kind of automate the testing, using some kind of a cloud service.

    But that’s not – if we go back to kind of what we also discussed in the beginning, if the goal is to accelerate kinda the entire engineering organization, there’s more to just running tests, you know? So every tech leader wants to – I mean, their objective is to deliver bug-free products to customers as quickly as possible. So we’re talking not just about testing but also configuration and delivery processes that often involve multiple technology stacks, which, of course, evolve over time. Now we have container technology, for example, which we didn’t have –

    Shimel: Changed the game.

    Anastasov: Completely changed the game. So the way to do this is to basically implement continuous delivery so that, step one, you automate all the processes and step two is to kind of optimize them, but using Semaphore for automating all continuous delivery processes has so far been – it wasn’t able to support, let’s say, every process that’s needed, so this is why, this November, we’re excited to present the brand new version of Semaphore, which is —

    Shimel: That’s Semaphore 2.0, right?

    Anastasov: Exactly. Yeah.

    Shimel: Mm-hmm. So, Marko, let me ask you this. I think you guys are pretty proud; on your website, you mention your bootstrap. You know, it’s interesting and, when you look at the CI/CD space, there’s a lot of great open-source, free tools, right? Jenkins, obviously, is the obvious one, but CloudBees, of course, has the commercial version, if you wanna call it. CircleCI, Travis – what’s the other one? – City – I forget the name of it. You and I both know there’s several pretty popular CI/CD tools, ARAs and all of this stuff. What do you think the special sauce is for Semaphore, if we’re talking – and our audience is developer-heavy, DevOps-heavy – why Semaphore versus maybe some of these others?

    Anastasov: Sure. We’ve always been focused a lot on speed, so “How can we not just provide a tool that automates something for people, but can we give them something preferably out-of-the-box, so that they just save time, comparing to whatever else they were using previously?” So one of the kind of foundational things, when you use Semaphore, is that basically we kind of periodically keep measuring this and it always stays true, is that Semaphore is twice as fast, comparing to all other competitors in the cloud space, in terms of just pure performance.

    So this alone can kinda save hours of engineering time to any team. Of course, it depends on the size of the projects and the team, but, if you kinda think back, kind of the fully-service hourly rate of developer in US is about $150.00. So, if you can do something to optimize that, we think it’s a huge win. Over time, we’ve kind of made steps forward, in terms of –

    [Phone ringing]

    Is this somebody from Jenkins calling to – [chuckles]

    Shimel: Yeah, it could be. [Laughs] That was pretty timely, but, no, I think it was something else. Go ahead.

    Anastasov: Yeah. So we provide tools that, for example, automatically paralyze the CI builds for projects, so people – which is, you know, large projects, for medium or bigger-sized teams with 100 or more developers, they typically have CI builds that run for hours. And the feedback loop – the kind of feedback loop in which you find yourself working, as a developer in such a project, it’s a pretty miserable experience. So you kind of become defensive. You work in wandering branches; you don’t merge off and – you know, you have flaky tests and so on. So, essentially, what we’ve seen, when people adopt Semaphore is, if they start using our automatic paralyzation feature, they go from, in some cases, from over 2 hours to something like 15-minute CI build. And it’s literally like they just needed to drag a slider and it worked. So we try to do things that really kind of improve the productivity of the teams.

    And so far, on the market also, it hasn’t been – in Semaphore 2.0, we’ve kinda took a lot of kinda learning that we picked up along the way and from our customers, especially the bigger customers, and, first of all, we made it really easy to create complex pipelines for companies that need to model the continuous delivery processes. On the other hand, what we think is pretty important is that, so far, this kind of cloud-based CI and CD tools – and Semaphore was part of this. In fact, we kind of initiated this idea of kind of charging per worker or per concurrent some kind of a machine that’s executing your stuff, your jobs.

    So, with Semaphore 2.0, we made the decision to kind of provide a service which is, let’s say, truly cloud, so it scales automatically to your needs, so you only pay what you use and you never really have any waiting time, which is a huge bottleneck we see in many organizations. They would maybe purchase some resources on a monthly or annual basis from Semaphore or Circle or some other product, and it’s like, if they are, let’s say, really busy in a certain period of a day or week, their CI just doesn’t process what they need. So we think that, with auto-scaling, developers will always be able to kind of get their work done quickly and stay in the zone as much as possible.

    Shimel: Got it. Hey, Marko, man, I would love to talk to you for another hour about maybe state of developers. And I do think you’re right – while speed is really important, as a benefit or an advantage to Semaphore, I think we’re in a world where developers are the top alpha predator, right? The most difficult kind of shoes to fill, the hardest jobs to get. Giving them an environment that keeps them happy, that keeps them working, if you will, in a good way, is a huge bonus than, frankly, making them work with crappy tools. I’m not calling anybody’s tool “crappy,” but I’m just saying, given the choice, I think developers would – they’d wanna work with tools that make their life easier and make it a better experience for them. But we’re at a half-hour and this was way more than 15 minutes, man, so we’re gonna need to end this. Maybe we can pick it up a second – in another chat in a couple weeks or whatever.

    Anastasov: I’d love to.

    Shimel: But, for now, we’ll call a wrap on this episode of DevOps Chat. Interesting conversation with Marko Anastasov. Close?

    Anastasov: That’s correct. Yeah.

    Shimel: All right. Cofounder of Semaphore. The new Semaphore 2.0’s coming out; check it out – great tool for CI. Until next time, this is Alan Shimel for devopsy.kinsta.cloud and you’ve just listened to another DevOps Chat.

    — Alan Shimel

  • Defining DevOps: How Do You Know When You Have Achieved DevOps?

    Defining DevOps: How Do You Know When You Have Achieved DevOps?

    Key to understanding where you are in your organization’s journey is by defining DevOps

    Much has been written about the compelling benefits of DevOps. However, the lack of a commonly agreed definition and clear prescription for implementing DevOps have left many people puzzled. Ask a group of experts, “How do you know when you have achieved DevOps?” and you are likely to get different answers.

    It is tempting to dismiss definitions as academic. However, there are many incomplete, floundering and costly DevOps implementations that fall short of expected business benefits. This article explains some of the key challenges of defining DevOps, and the importance for having a definition at least within the scope of an enterprise. This article includes suggestions to help enterprises navigate their DevOps journey toward successful business outcomes.

    Defining DevOps

    An enterprise needs to know when it has accomplished DevOps to justify investments that support business goals. Pinning down a universal definition that satisfies the diverse community of stakeholders and survives the test of time has proven elusive. DevOps spans an evolving body of knowledge covering multiple humans, process engineering and technology disciplines. DevOps is never really “done” because the highest level of maturity, “The Third Way,” continuous improvement, defies setting boundary conditions.

    To reconcile this dilemma, some have defined DevOps using vague or transient terms such as a “cultural movement.” Others have skirted the problem of definition with rambling statements describing what is involved to do DevOps, rather than state what it is in concise and concrete terms. Despite these challenges, acceptance of a definition by enterprise stakeholders is a key first step in any successful DevOps transformation. Any meaningful project progress measurements hinge on clear definitions.

    I have found the following DevOps definition useful, relatively concise, and stable: “DevOps is the application of continuous flow, feedback and improvement to people, processes and technology for the benefit of agility, stability, efficiency, quality, security, availability and satisfaction.” Defining DevOps in this way is broad enough to stand the test of time, yet specific enough to answer key questions addressed by a good definition including “What is it?” “How it is done?” and “Why do it?” without too much rambling.

    Whether you create of choose another definition, or decide to use mine, it is important that your cross-functional teams agree on one to align their goals and implementations and to have a basis to measure your DevOps progress and business outcomes.

    DevOps and Continuous Delivery Partnership

    DevOps is often said to refer to cultural aspects of high-performance software development and operations including leadership, teams and collaborative practices. Nevertheless, technical mechanisms involved in architecting applications and toolchains are essential for continuous delivery pipelines. Culture, without the technology associated with continuous delivery pipelines, will not achieve the expected benefits of DevOps and vice versa. Therefore, a practical measure of DevOps maturity really must include the technology and not only the culture.

    Continuous Flow is a Practical Baseline

    DevOps benefits are derived primarily from applying lean manufacturing methods to software development and operations. The “First Way of DevOps,” continuous flow, is first because it is a prerequisite to the Second and Third Ways. Therefore, the realization of continuous flow for at least one application in an enterprise can be considered a baseline for “having DevOps.” If you accept this point of view, then the answer to the question, “How do I know when I have DevOps” can be equivalent to the question, “What is the minimum requirement for continuous flow for an application?” In the context of continuous delivery toolchains, a continuous flow pipeline is achieved when it allows code submissions from development to successfully transit all pipeline stages to deliverable packages acceptable for production deployment.

    A DevOps Model Application

    Enterprises typically have many applications moving toward DevOps at different paces. Identifying one application as a model for DevOps is useful. The model provides a focal point for DevOps investments, and experiments. Other application product teams learn from the model application as it matures from discontinuous flow to continuous flow and higher levels of DevOps maturity.

    Summing Up

    A practical answer to, “How do you know when you have DevOps” depends on having a definition. While defining DevOps itself has been elusive, an enterprise definition is needed for alignment and progress measurement. Considering that DevOps needs continuous flow to accomplish business goals, it can be said you have DevOps when you have implemented continuous flow for at least one model application.

    — Marc Hornbeek

  • CloudBees Further Extends Jenkins CI/CD Reach

    CloudBees Further Extends Jenkins CI/CD Reach

    CloudBees announced it is expanding the scope and reach of its Jenkins open source continuous integration/continuous development (CI/CD) platform into the realms of compliance and serverless computing frameworks.

    Announced at the DevOps World | Jenkins World conference in Nice, France, a new plugin for Jenkins adds support for the AWS Serverless Application Model (SAM) defined by Amazon Web Services (AWS). That model spans AWS Lambda functions, application programming interfaces and Amazon DynamoDB tables and is natively supported by AWS CloudFormation. The AWS SAM plugin supports both classic FreeStyle jobs and Jenkins Pipeline to make it possible to read and deploy serverless applications.

    In effect, CloudBees is making it possible to treat AWS Lambda as a natural extension of any CI/CD pipeline. Longer term, serverless computing frameworks will play a larger role in the cloud-native instance of CloudBees as well.

    In the meantime, CloudBees is also extending its CI/CD platform to address compliance issues that arise from identity management. CloudBees Core is now integrated with the LDAP-based directory Google makes available within its Cloud Identity service. This capability makes it possible to centralize user management policies by using the same Google Cloud Identity or G Suite credentials to connect to CloudBees Core.

    Christina Noren, chief product officer for CloudBees, noted governance and compliance are becoming much bigger issues within best DevOps practices as more responsibility for security continues to shift left toward developers.

    The CloudBees community continues to grow, even as the number of CI/CD rivals continues to expand. At the conference, CloudBees executives revealed the Jenkins community has grown 24 percent in the period Sept. 30, 2017, to September 30, 2018, reaching 192,623 active installations. That number only applies to installations that permit usage information to be transmitted to the Jenkins project, so this number is not representative of the total Jenkins user population. All told, it’s estimated there may be anywhere from 15 million to 23 million developers globally relying on Jenkins. CloudBees also noted the combined number of defined Jenkins Pipeline and other jobs increased during this same annual period, from 10,509,706 million to 20,664,054 million, with Jenkins Pipeline jobs demonstrating growth of 309 percent from last year.

    Noren said one of the primary reasons for that continued success is that unlike other CI/CD platforms that provide a highly opinionated framework accessible only as a cloud framework, the Jenkins environment makes it possible to deploy a more flexible framework on-premises or in a public cloud.

    Of course, there are still millions of developers that have yet to be exposed to any type of CI/CD framework. But as DevOps practices continue to become the primary method through which application development and deployment is increased, it’s now only a matter of time before most developers are working within the context of a CI/CD platform.

    In fact, it’s arguable that those organizations that don’t soon embrace CI/CD platforms will be overwhelmed by rivals capable of updating applications at rates that are increasingly being measured in days and hours versus weeks and months.

    — Mike Vizard

  • Electric Cloud Extends Continuous Delivery Platform

    Electric Cloud Extends Continuous Delivery Platform

    Electric Cloud is making available a software-as-a-service (SaaS) pricing option available for its ElectricFlow application release management and continuous delivery (CD) platform.

    In addition, version 8.5 of ElectricFlow adds support for a Kanban-style pipeline view, object tagging for custom reporting and improved searchability and a continuous integration (CI) dashboard to track and analyze build processes, failures and successes.

    The SaaS option doesn’t fundamentally change the existing Electric Cloud pricing model as much as it does provide a different location to host ElectricFlow, said Electric Cloud CTO Anders Wallgren. Pricing for ElectricFlow is still based on nodes and the number of users rather than consumption.

    The CI dashboard, meanwhile, provides organizations that have one or more types of CI platforms being employed with visibility into, for example, Jenkins, Git and Bamboo pipelines using a single dashboard. Many organizations today have multiple CI pipelines in place largely because of the preferences of individual developer teams, Wallgren noted.

    The Kanban-style pipeline also provides a mechanism for managing multiple application development projects based on a just-in-time process originally developed by Toyota that is employed to identify bottlenecks in car manufacturing. That same issue, of course, applies to software development, where the number of dependencies between various modules is increased exponentially. That issue is becoming even more pronounced as organizations move to build and deploy cloud-native applications based on various types of runtimes. DevOps teams now can approve and observe release progress and deployment status within ElectricFlow in near real time, Wallgren said.

    Finally, tagging functionality on all objects allows release managers and product owners to create more granular, real-world reports quickly and easily. Universal data segmentation replaces code-based report creation and can be applied across any object type.

    Wallgren noted that adoption of DevOps best practices generally continues to advance at uneven rates. Electric Cloud is now working closely with the DevOps Institute on an online training and certification program dubbed Electric Cloud University. The goal is to increase the number of IT teams that are exposed to modern CD processes at whatever pace they choose. Electric Cloud is making four certifications available: Build Accelerator, Test Accelerator, Administrator and Agile Master.

    Wallgren said the next major frontier when it comes to continuous delivery will be applying deep- and machine-learning algorithms to tame what has become an exceedingly complex management challenge. DevOps toolchains continuously create vast amounts of data originating from numerous tools, and machine learning algorithms can harness that data to automate a range of DevOps processes in a way that is not only makes those processes more accessible, but also easier to demonstrate how an application release meets a specific compliance mandate. To achieve that goal, Electric Cloud has been embedding more algorithms into every release cycle of ElectricFlow, says Wallgren.

    It may take a while longer for algorithms to minimize all the complexity associated with continuous delivery today. But as DevOps continues to evolve, it’s apparent that many of the manual tasks that have often conspired to limit adoption of DevOps will one day soon disappear.

    — Mike Vizard

  • SnapLogic Extends DevOps Reach via GitHub

    SnapLogic Extends DevOps Reach via GitHub

    SnapLogic has extended the reach of its SnapLogic Enterprise Integration Cloud into the realm of DevOps by making it possible to store integrations creating using Snap connectors in GitHub.

    The company also announced integration with the DC/OS platform from Mesosphere along with a new service catalog and several updates to the Iris artificial intelligence (AI) software it uses make recommendations concerning the construction of integration pipelines. At the same time, SnapLogic has enhanced both its monitoring tools for application programming interfaces (APIs) and search tools for discovering broken integration pipelines.

    Craig Stewart, vice president of product management for SnapLogic, said support for GitHub makes it easier for developers at all skill levels to share preconfigured integrations via GitHub alongside all the other code they already share via the widely employed repository. That capability also makes it easier for DevOps teams to include preconfigured Snap connectors within a continuous integration/continuous development (CI/CD) pipeline, he noted, adding those integrations now can be more easily discovered within a Patterns Catalog to help developers discover and reuse integration patterns.

    Support for Mesosphere, Stewart said, is being provided by customer request. That effort will then be extended to include support for Kubernetes, which SnapLogic expects will become the de facto standard for container orchestration engine.

    Finally, SnapLogic updated its Iris AI software to provide an additional option for users to build their integration workflows starting with the last Snap of the integration pipeline. Previously, the Iris recommendation engine started with the first Snap connector. Now, when a user first places the last Snap on the canvas, the Integration Assistant will display suggested previous Snaps until the pipeline is complete. Users also have the option to start with the first and last Snaps of the pipeline and the Integration Assistant will work from the outside-in to recommend Snaps to build out the entire pipeline.

    SnapLogic is making investments designed to facilitate self-service capabilities that promote reuse of integrations, Stewart said. A recent survey of 500 IT decision-makers conducted by Vanson Bourne on behalf of SnapLogic finds on average organizations expect to generate a 547 percent return on their data investments, increasing revenue by an average $5.2 million by using data more effectively. But the survey also finds that on average organizations are using only half (51 percent) the data they collect or generate, and data drives less than half (48 percent) of the decisions being made. Organizations will spend $1.7 million on an average to operationalize data over the next five years to rectify that problem, according to the survey results, which is more than double what they currently spend.

    Much of that effort will be driven by so-called citizen integrators that reside within various lines of business outside the traditional IT department. In many cases those citizen integrators will be leveraging prebuilt integration patterns created by developer working within the internal IT department. The challenge now is finding a way to make sure all those integration patterns can be slipstreamed into a larger set of DevOps processes that are increasingly spanning the entire enterprise.

    — Mike Vizard