Tag: Jenkins

  • Jenkins User Conference and CD Summit: Calling all London DevOps

    Jenkins User Conference and CD Summit: Calling all London DevOps

    I wanted to give a quick shout out to the large, vibrant and supportive London DevOps community. Later this month (June 23rd and 24th), there will be a joint conference at the Hilton Metropole, London. Combining the Jenkins User Conference and Cloudbees CD Summit, it is two days of CD/CI/DevOps learning.

    Speakers include:

    Kurt Bittner, Principal Analyst Forrester,

    Martin Croker, Managing Director, DevOps, Accenture,

    Sanjeev Sharma, CTO, DevOps Technical Sales & Adoption, IBM Distinguished Engineer, IBM,

    Viktor Clerc, Product Manager, XebiaLabs

    and more.

    The Jenkins User Conference (JUC) features another great lineup of speakers as well.  devopsy.kinsta.cloud as well as a host of others are sponsoring the event.

    In speaking to my friends at Cloudbees they mentioned that there were tickets still available. I found that hard to believe as I know what the community is like here in London.  So if you are going to be in town on the 23rd and 24th, don’t miss this event!  I will be there hosting a great panel on the 23rd as well.

    The Jenkins community is one of the most active that I have ever seen. I am looking forward to seeing many of you in London on the 23rd ad 24th.

    There are still sponsorships for CD Summit as well.  If interested you can contact Cloudbees for info.

    Also this Thursday is DevOps Connect: Rugged DevOps @Infosecurity Europe all day. Registration is free, so come down for a great day of DevOps tracks!

  • 31 Reference Architectures for DevOps and Continuous Delivery

    31 Reference Architectures for DevOps and Continuous Delivery

    At QCon London, David Farley (@davefarley77) told the audience that “continuous delivery changes the economics of software delivery”.  I could not agree more.

    If you have been drawn to the evangelists like David Farley, Jez Humble, and Gene Kim, you’ll know that high performance IT organizations are seeing a massive payoff in their continuous delivery investments.  Industry leaders big and small are sharing noticeable results:

    • 8x more frequent production deployments
    • 50% lower change failure rates
    • 12x faster service restoration times when something went wrong

    5 Principles

    How are they achieving these results?  They are following a few key principles that David and Jez outlined in their seminal book, Continuous Delivery (highly recommended).  The principles outlined in their book and in David’s QCon presentation are as follows:

    • Deliver fast
    • Automate almost everything
    • Keep everything in version control
    • Build quality In
    • Empower the team 

    Reference Architectures for DevOps and Continuous Delivery

    In their book, they also provide insight to continuous delivery tool chains and processes which highlight build tools, CI platforms, testing suites, artifact repositories, and many other components. There are numerous examples of reference architectures available, and each of them vary in levels of detail, tools highlighted, and processes followed.  Yet, there is a constant theme among the tool sets: Jenkins, Maven, Nexus, Subversion, Git, Docker, Puppet/Chef, Rundeck, and Sonar seem to show up time and again.

    To help you along your continuous delivery and DevOps journey, I have compiled a set of 31 reference architectures created by users across the continuous delivery and DevOps communities.

     CD and DevOps tool chain

    Each architecture is accompanied by a link to the original presentation or blog where the architecture was referenced to ensure you have access to the full context of the discussion. You can also freely download the presentation from SlideShare where we now have over 7,000 views.

    Offer to help others: If you have a reference architecture to share, please consider posting a link to it in the comments section below.  I am sure others would love to see it and learn from it.

  • Chef defines the DevOps workflow with Chef Delivery

    Chef defines the DevOps workflow with Chef Delivery

    At ChefCon today Chef made a major announcement unveiling Chef Delivery a “new DevOps workflow product that for the first time enables the continuous delivery of infrastructure, runtime environments – including containers – and applications.”  Chef has examined the use success patterns of many of its thousands of customers to distill what they believe is the formula for success in a DevOps environment.

    alex ethierI had a chance to speak with Alex Ethier, VP of Product and Jay Wampold, VP of Marketing at Chef about Delivery. They were both really jazzed as they think this is a real game changer for not only Chef but the entire DevOps market.  Chef is already a leader in the DevOps community with a rapidly growing base of customers in the Global 2000 and a dominant adoption position among AWS customers. Delivery will extend the Chef platform into the entirety of the DevOps workflow.

    Chef believes this is the time to expand the Chef platform to the entire DevOps workflow. Just about every analyst firm and survey data indicates that DevOps is on the way to wide spread adoption among IT teams of all sizes. While DevOps is about cultural change, once the organization recognizes the advantages to a DevOps culture, tools that leverage this culture are sure to expand their footprint.

    I asked Alex and Jay if Delivery would bring Chef into competition with other DevOps tool vendors. For instance Jenkins and CloudBees in the CD and CI area would seem to be square in the cross hairs of Delivery. While on its surface this would seem to be the case, Delivery is actually a platform that is designed to leverage APIs and open source to to work with a large ecosystem of 3rd parties including Jenkins, CloudBees and other players in the DevOps ecosystem.  I am still not 100% sure where the cooperation will end and the competition will begin, but I guess we will have to wait and see how that shakes out.

    Based on Chef’s customer experience research they think this is what the flow of DevOps looks like:

    chef pic
    DevOps lifecycle step by step

    Some steps are and will remain manual. Chef is seeking to play in the automated areas. Yes this is an ambitious move by Chef. If they succeed they will leverage their already dominant position in the DevOps configuration management field into becoming a dominant player in the entire DevOps life cycle.

    Powering this life cycle is the fact that with the move to the cloud today’s enterprises are moving from static systems to rapidly changing services. This is putting new pressures on IT departments to perform in a “high velocity IT” manner. The key to all of this is of course automation. Under it all Delivery seeks to automate CI, CD and continuous testing.

    Chef’s CTO, Adam Jacob had this to say about Chef Delivery, ““The tools we use reinforce the behavior; the behavior reinforces the tool. Thus, if you want to change your behavior, change your tools.”

    Other highlights of Chef Delivery from the Chef press release are:

    • Robust Pipeline: Chef Delivery provides a shared pipeline and proven workflow for software development that safely takes code from a developer’s or operations engineer’s workstation through build, test, and production.
    • Platform for Collaboration:  Each step of the Delivery pipeline incorporates automated testing to provide developers, operations engineers, compliance and security officers, and IT architects rapid feedback on proposed changes. The common workflow allows personnel of each IT discipline to work together with full visibility into the flow of changes through the pipeline. Policies can be easily applied at each step to ensure maximum change control and governance.
    • Sophisticated Analytics: Chef Delivery provides metrics for all stages of your development pipeline. With full audit capabilities, you can track both pipeline performance and activity, manage permissions and access comprehensive change history from easy to use dashboards.
    • Scalable Architecture: Chef Delivery was designed from the ground up for performance and scalability. Its architecture is fully integrated with the Chef Server and designed to meet the demands of the largest and most complex enterprise IT environments.
    • Full Ecosystem Integration: Chef Delivery integrates with an extensive array of operating systems and runtime environments, including public cloud platforms such as Amazon Web Services and Microsoft Azure, as well as runtime environments such as Docker containers.
    • Availability: Chef Delivery is currently available through an invitation program
    • Pricing: Chef Delivery will be sold in 2015 on a subscription basis.

    This could indeed be a real game changer for already high flying Chef.  While at ChefCon today I intend to ask many of the partners and vendors exhibiting here how they view Delivery and what kind of integrations we can expect to see from them for it.

    One thing for sure is the DevOps tools market is evolving and maturing rapidly.  Tool vendors will have to move quickly to allow high performing IT teams to move at the speed of DevOps.

  • CloudBees Raises $23.5 Million

    Funding round to fuel the continuous delivery leader’s significant expansion in 2015

    LOS ALTOS, CA.  – January 27, 2015 – CloudBees, Inc., the Enterprise Jenkins Company and continuous delivery leader, today announced a $23.5 million financing round, led by longtime CloudBees investor Lightspeed Venture Partners. Existing investors Matrix Partners, Verizon Ventures and Blue Cloud Ventures also participated in the round.

    The latest funding announcement follows earlier rounds of $4 million, $10.5 million and $10.8 million, bringing the total investment in CloudBees to just under $50 million since the company’s inception in 2010. Previous venture investment rounds were led by Matrix Partners, Lightspeed Venture Partners, Verizon Ventures with participation from Blue Cloud Ventures, JBoss alumni Bob Bickel and Marc Fleury, and Ed Bugnion, co-founder of VMware.

    CloudBees plans to use the funding to facilitate the ongoing and significant growth the company is experiencing from demand for its continuous delivery (CD) solutions based on Jenkins CI, the leading open source continuous integration (CI) platform. With Jenkins’ global dominance as the hub of continuous delivery for enterprises, CloudBees sees explosive demand in 2015 for its Jenkins-based solutions. The CloudBees solutions enable organizations to leverage Jenkins and CD practices at enterprise scale.

    “This investment in CloudBees tops a year of continuous growth and acceleration, fueled by the adoption of continuous delivery as companies in all markets realize that software is eating the world,” said Sacha Labourey, founder and chief executive officer of CloudBees. “We will use the additional capital to serve the growing demand for Jenkins-based solutions worldwide, to continue to capture market share and to further cement our position as the continuous delivery leader.”

    CloudBees plans to make investments in a variety of areas – including development, marketing, sales and professional services  – to increase marketshare. The company also plans to introduce new products to serve an even broader range of customers – from smaller implementations to full-scale enterprise footprints – and to make new investments in strategic partnerships.

    Growth in the enterprise uptake of Jenkins has been an important business driver for CloudBees and was a primary factor in its successful strategic shift in 2014 to focus solely on continuous delivery solutions powered by Jenkins. Global brands that subscribed to CloudBees’ enterprise offerings in 2014 include Amadeus, 3M, TIAA-CREF, Intuit, Alcatel-Lucent, Discover Financial Services, EMC, Fidelity, Lufthansa, Fannie Mae, Amdocs, The Federal Reserve, Hyatt Hotels, Disney, PG&E, Northrup Grumman, USDA, Robert Bosch, EMC, Verizon, ABN AMRO, Nordstrom, Cisco and T-Mobile.

    “We’ve been with CloudBees since the early days and we continue to be impressed with the way this world-class team delivers on its market vision,” said John Vrionis, general partner with Lightspeed Venture Partners. “The business is scaling quickly and we believe there is a massive opportunity to build a franchise company, given all the momentum in continuous delivery.”

    Additional Resources

    About CloudBees
    CloudBees (www.cloudbees.com), the Enterprise Jenkins Company, is the continuous delivery (CD) leader. CloudBees provides solutions that enable IT organizations to respond rapidly to the software delivery needs of the business. Building on the strength of Jenkins CI, the world’s most popular open source continuous delivery hub and ecosystem, the CloudBees Continuous Delivery Platform provides a wide range of CD solutions for use on-premise and in the cloud that meet the unique security, scalability and manageability needs of enterprises. The CloudBees Jenkins-based CD solutions support many of the world’s largest and most business-critical deployments.

    Backed by Matrix Partners, Lightspeed Venture Partners and Verizon Ventures, CloudBees was founded in 2010 by former JBoss CTO Sacha Labourey and an elite team of continuous integration, continuous delivery and DevOps professionals. Follow CloudBees on Twitter, Facebook,LinkedIn and Google+.

  • 15 must have Jenkins plugins to increase productivity

    15 must have Jenkins plugins to increase productivity

    Jenkins is a vastly used continuous integration tool for the IT industry which is built and used around plugins. It’s core is the Jenkins tool and there are 100’s of plugins available to enhance its power and usability.

    Let’s take a look at a few of the best plugins to increase the productivity.

    1. Global Build Stats Plugin–It’s essential to know your current capacity, usage and capability to prepare for capacity planning or system requirements, because the very first question is “what is your current capacity?”         global-build-stats

    You need to know how many builds are happening on a daily and weekly basis. How much time is taken by various builds, what is the waiting period. This plugin give you enough data to answer the question. This plugin can provide you data as graph which can be further consumed.

    1. Job GeneratorPlugin–In big or growing organizations, its a bit difficult to maintain the jobs for a project when developers are working on various branches and releases. You want to give access to developers to create their own job and at the same time you don’t want developers to create arbitrary jobs which may not fall under company standards. This plugin gives you flexibility to define templates and developers can create new jobs with the help of job generator template. Configuration access can be disabled via roll-based authorization plugin.
    2. Disable-failed-job –In rapid development environments failing jobs due to various unnecessary reasons are a pain, It’s difficult to keep track of such jobs and disable/delete them. This plugin can solve the problem by defining the upper number of consecutive failed builds and then disable automatically.
    3. Embeddable-build-status –This plugin can give you a link which can be pasted anywhere (example github project) to expose the status of the build and users can get the current state of the job while looking at project.
    4. Exclusion –This plugin enables you to handle conflicts between jobs. You can assign a resource (or lock) in multiple jobs, when build is executed, it will acquire the lock and other builds (if fired) will wait until lock is released.
    5. GitHub/GitLab Pull Request Builder-This is one of the best plugins I found to provide support to automate code review up to a certain extent in github/gitlab. This plugin does amazing job once defined for the project. For any new pull request this plugin does fox merge and runs the build on the code as well as gathers the necessary static analysis (if configured) or build results and comment back the status to pull request. This helps reviewers get an idea of the health of the code which is going to be merged. You can even define automatic merge if build passes.
    6. Hudson Extended Read Permission Plugin- During initial days of Jenkins setting configuration access was not available to any developer to make sure they do not change anything but developers used to ask to view configurations to know how job has been setup, they may want to see the build steps. Extended read plugin can provide option to developer to view the configuration without giving them write access.
    7. Post+build+task –There may be a need for performing some actions on the basis of the results of a build, for example if build passed you may want to upload artifact(ex debian) to some repo (apt) or perform some packaging part or similar. In case of failure you may want to roll back something (like release) . This plugin helps you to define the pass/fail criteria and let’s you decide what to do after that.
    8. JDK Parameter Plugin–This plugin is useful for an organization where many projects are using different versions of java. This plugin let’s you choose the java version during run time of the build.
    9. Job Configuration History Plugin-Ah! This is one of my favorite plugins. This plugin lets you keep track of config changes in each build including who did it. You can easily revert back to any previous config if you want.
    10. Multiple SCMs plugin–Default SCM section provides only one source control tool/URL option, what if you want to check out from more than 2 repos from multiple source control tools (like svn and git). This plugin will come in handy in such scenarios. This plugin will facilitate users to add any number of SCM URLs to checkout the code.
    11. Parameterized Trigger plugin– Another one of my favorite plugins. This plugin allows you to have user input as a variable and use on run time. This is the most used plugin in dynamic environments where you have lots of options and user-defined values to be used in the build which may keep changing.
    12. Pre SCM BuildStep Plugin- Just like post build task plugin, you may have requirements to perform some action even before checkout happens for the job, for example you may want to perform merging of the branch before the build and then checkout. This plugin can come in handy in various conditions and gives you flexibility when running the job.
    13. SCM Sync Configuration Plugin-backup, this is the most important task of any administrator. Without regular backup, the whole system cannot be reliable; this plugin provides you features to backup live Jenkins configs to any source control tool. It will keep committing the config files (including Jenkins and jobs) to SCM repository as soon as there is any change.
    14. Configuration Slicing Plugin – This plugin comes  in very handy when you want to make bulk changes in multiple jobs. This plugins allows you to change values of various fields like email, timer, shell script, configurations etc at one go.

    These are my favorites, but like I wrote, there are 100’s of plugins for Jenkins available.  What are some of your favorites?

  • DevOps Transforms Copyright Clearance Center

    DevOps Transforms Copyright Clearance Center

    Copyright Clearance Center (CCC, www.copyright.com) works primarily with business, academic, and medical institutions, enabling them to access the vital content that supports their work. CCC manages fee payments to the license holders and content creators who own the content. DevOps approaches have transformed CCC, a global leader in content rights-licensing solutions, resulting in expanded service to an increasing market.

    “In production, CCC is already processing millions of dollars in transactions. This came to life within a year of the inception date for the DevOps project. For a startup that would be fast. For a medium-sized enterprise, I think that is outstanding,” says Babis Marmanis, CTO & VP of Engineering, Copyright Clearance Center.

    The DevOps transformation that took place in the internal structure of CCC’s engineering department continues today, allowing the organization to respond to the massive disruption in the publishing industry, which is due in part to digital content and the Internet. Using DevOps, Marmanis and CCC have accelerated product development, sped time to market, and provided a stronger consumer experience with the product.

    Accelerated Product Development

    “The Copyright Clearance Center vision is to create the most comprehensive portfolio of global licensing and content solutions (products) that make copyright work for everyone. My product platform strategy stems from that,” says Marmanis.

    CCC has multiple software-based products, each performing the same or similar functions and acting on the same or similar data. “There are certain fundamental things that we must abstract appropriately and encapsulate into common components to make development work optimally,” says Marmanis.

    But when Marmanis came to CCC, there were no common components. And what is more the internal systems lacked interoperability, skills were different across internal teams, and there was no cohesiveness to the development approach. These factors slowed software product development.

    A DevOps mindset was crucial to changing the status quo, to treating infrastructure as code, and to bringing in automation, says Marmanis. A DevOps approach would help to speed the delivery of software projects into the marketplace.

    “Now we use Jenkins (www.cloudbees.com) to glue together elements of the continuous integration process. We have Jenkins build jobs that retrieve the source code from GitHub (https://github.com). We publish software artifacts to Artifactory (www.jfrog.com),” says Marmanis.

    For each release, Marmanis’s teams use Jenkins to execute Chef (www.chef.io) recipes. They deploy artifacts from Artifactory onto a pre-specified execution environment. “In addition, and this is key to the overall scheme, we execute the schema updates with Liquibase (www.liquibase.org), and everything is coordinated with Jenkins,” says Marmanis.

    Further, CCC built its cornerstone platform from the ground up using DevOps-related tools, build detection software, and software libraries. The infrastructure that CCC added is based on best-of-breed open source libraries.

    “Using programming languages such as Java (www.java.com) and Groovy (http://groovy.codehaus.org), Gradle (www.gradle.org) scripts, postgreSQL as the relational database, Liquibase for database objects, Tomcat (http://tomcat.apache.org) as a servlet container, Apache Camel (http://camel.apache.org) for messaging and enterprise integration, and countless other tools including Apache Directory Studio (http://directory.apache.org/studio) and Bootstrap (http://getbootstrap.com/)—the list is very long—we were able to create a harmonious ecosystem,” says Marmanis.

    Stronger Consumer Product Experience

    CCC offers its consumers a better product experience through DevOps approaches and through a homogeneity of standards and technologies, which is available when using open source tools.

    “When we’re building the product platform, these initiatives allow us to embody timeless design principles such as crisp abstractions, but also allow us to scale in a number of dimensions including the number of works that we can represent, the number of publishers we can admit and manage, the number of customers that we have, the number of applications that we can build and deploy, and the number of market verticals that we can occupy,” says Marmanis. This is crucial to customer satisfaction.

    One product that is a prime example of how CCC keeps customers happy is RightsLink for Open Access. “Using RightsLink for Open Access, we facilitate fee collection for articles published under an Open Access license, mediating between the publisher and the author to collect those fees,” says Marmanis. Then CCC aggregates those transactions and presents them to the publisher.

    “Obviously, we give the money to the publisher, but we also create reporting for the authors, publishers, and institutions as well as the founding agencies that gave the money to the author or the institution respectively for funding that research,” says Marmanis. The RightsLink for Open Access product keeps transactions simple, integrating with publisher workflows to expedite publication.

  • Breaking Builds II – The Butler did it

    Breaking Builds II – The Butler did it

    reakingbadAs editor-in-chief here at devopsy.kinsta.cloud and writing for several other publications I see more than my share of contests, promotions and other marketing campaigns (not to mention pitches from PR agencies), but I came across one this week that I really wanted to share with you.  My friends at CloudBees, the Jenkins company are once again running their Breaking Builds promotion.  In case you don’t know about it, it combines two very different characters. The Butler from Jenkins and Walter White from Breaking Bad. As Sacha Labourey, CEO of CloudBees says in his blog post on the contest morphing these two together is “probably one of the best examples of cognitive dissonance you can get.”

    The marketing and engineering teams at CloudBees have come up with a series of banners with graphics of the Butler as Breaking Bad characters, especially Water White.  They then have come up with captions for the images that are spoofs of lines from Breaking Bad and also have a Continuous Delivery angle to them.  You can see what they have come up with here. Your mission if you accept it is to take the graphics and come up with a better caption than the CloudBees team. You then tweet the image with your caption and you’re entered.

    There are prizes for several categories of captions. The categories are

    • Most Original Caption for any of the 10 #BreakingBuilds images + your caption
    • Most Humorous Caption for any of the 10 #BreakingBuilds images + your caption
    • Most ReTweeted Twitter post* for any of the 10 #BreakingBuilds images + your caption *Dare we call this CI (Continuous Interaction)?  😉
    • Most Favorited Twitter post for any of the 10 #BreakingBuilds images + your caption
    • We may also create additional categories during the contest if we see an emerging theme!

    The categories, prizes and full rules are also available here.

    Personally my favorites of the images and captions done by CloudBees are these two:

    breaking-builds-say-my-namebreaking-builds-done-when-i-say

    So if you are a Jenkins and/or a Breaking Bad fan why not join in the fun. Give it a try. We will also feature some of the winning banners here on devopsy.kinsta.cloud

  • CI and CD Across the Enterprise with Jenkins November 19th

    CI and CD Across the Enterprise with Jenkins November 19th

    We recently held another webinar in our continuing series at devopsy.kinsta.cloud. We partnered with the good people at CloudBees on CI and CD across the enterprise with Jenkins. The webinar featured:

    Cyrille Le Clerc  is Director of Product Management at CloudBees with more than 15 years experience in Java.  Prior to CloudBees, Cyrille was Chief Technology Officer at Xebia.  As an early adopter of  the “you build it you run it” model, he put it in place for a number of high volume web sites.  He naturally embraced DevOps culture.  He is also very active in the Java community, creator of the open source project “embedded jmxtrans” and a speaker at various conferences.

     

     

     

    It proved to be the most popular webinar we have held yet with almost 1500 people registered! Many were not able to log in live for the event. We are making a recording available for those who could not attend and anyone else who would like to listen:

    Hope you enjoy it and stay tuned for more webinars.

  • CloudBees nurtures the Jenkins community to new highs

    CloudBees nurtures the Jenkins community to new highs

    One of the linchpins in the DevOps tools box is Jenkins.  Just about everyone knows Jenkins. For those few not familiar with Jenkins, here is an excerpt from the “What is Jenkins” page:

    What is Jenkins?

    Jenkins is an award-winning application that monitors executions of repeated jobs, such as building a software project or jobs run by cron. Among those things, current Jenkins focuses on the following two jobs:

    • Building/testing software projects continuously, just like CruiseControl or DamageControl. In a nutshell, Jenkins provides an easy-to-use so-called continuous integration system, making it easier for developers to integrate changes to the project, and making it easier for users to obtain a fresh build. The automated, continuous build increases the productivity.
    • Monitoring executions of externally-run jobs, such as cron jobs and procmail jobs, even those that are run on a remote machine. For example, with cron, all you receive is regular e-mails that capture the output, and it is up to you to look at them diligently and notice when it broke. Jenkins keeps those outputs and makes it easy for you to notice when something is wrong.

    Jenkins offers the following features:

    1. Easy installation: Just java -jar jenkins.war, or deploy it in a servlet container. No additional install, no database.
    2. Easy configuration: Jenkins can be configured entirely from its friendly web GUI with extensive on-the-fly error checks and inline help. There’s no need to tweak XML manually anymore, although if you’d like to do so, you can do that, too.
    3. Change set support: Jenkins can generate a list of changes made into the build from Subversion/CVS. This is also done in a fairly efficient fashion, to reduce the load on the repository.
    4. Permanent links: Jenkins gives you clean readable URLs for most of its pages, including some permalinks like “latest build”/”latest successful build”, so that they can be easily linked from elsewhere.
    5. RSS/E-mail/IM Integration: Monitor build results by RSS or e-mail to get real-time notifications on failures.
    6. After-the-fact tagging: Builds can be tagged long after builds are completed.
    7. JUnit/TestNG test reporting: JUnit test reports can be tabulated, summarized, and displayed with history information, such as when it started breaking, etc. History trend is plotted into a graph.
    8. Distributed builds: Jenkins can distribute build/test loads to multiple computers. This lets you get the most out of those idle workstations sitting beneath developers’ desks.
    9. File fingerprinting: Jenkins can keep track of which build produced which jars, and which build is using which version of jars, and so on. This works even for jars that are produced outside Jenkins, and is ideal for projects to track dependency.
    10. Plugin Support: Jenkins can be extended via 3rd party plugins. You can write plugins to make Jenkins support tools/processes that your team uses.

    But beyond the wiki how did Jenkins become one of the most successful open source projects of all time? The answer to that has a lot to do with CloudBees, a little to do with Oracle and even more to do with an architecture that supports an ecosystem that has allowed almost a thousand plugins to be developed to the core project.

    First let’s get the little do with Oracle out of the way.  Jenkins was originally known as Hudson. Hudson was started by Sun Microsystems. When Oracle bought Sun it inherited a lot of open source projects (OpenOffice and Java among them). Many in the open source community questioned Oracle’s intentions regarding open source. It didn’t take long for a dispute to arise between many of the Hudson developers and Oracle. Oracle claimed they owned the term Hudson and that had an easy answer. There was a vote taken and Hudson became Jenkins. When Oracle objected, Jenkins became a fork of Hudson (or Hudson became a fork of Jenkins). Oracle still maintains Hudson, which is a shell compared to Jenkins.

    In the meantime Sacha Labourey former CTO at JBoss, started CloudBees in 2010. This was after JBoss was acquired by Red Hat. CloudBees created a PaaS that was one of the first PaaS to support the entire application lifecycle from development to deployment. A key part of CloudBees offering was the Jenkins Continuous Integration (CI) engine. Labourey brought in Kohsuke Kawaguchi, a founder and key contributor to Jenkins. Along with Kawaguchi, several other key Jenkins contributors also came to CloudBees.

    CloudBees and their employees contributed more than 80% of the code to Jenkins. Jenkins and the community that supports it is thriving. I had a chance to speak with Kawaguchi about this. He is very proud that there are over 1000 plugins now for Jenkins. I was blown away by a number of something like 500k downloads a month from the Jenkins website.

    Of course a thriving open source community does not necessarily translate into a thriving commercial business. There have been many companies that never cracked the formula to monetize there open source success.

    This is where CloudBees appears to be winning. I spoke to Kwaguchi about CloudBees commercial success. With offices around the world CloudBees has been able to translate an iconic open source project into a thriving business.  They have a set of enterprise (ready pay for) plugins that enhance Jenkins in a number of ways. Additionally, the offer hosted versions of Jenkins, CI-a-a-S if you will. They call in Jenkins Operations Center. They also offer Dev@cloud, which is an umbrella for other services and software that enhance and make Jenkins easier to use across different platforms, with different languages and other options. Of course they offer support for your Jenkins installs as well.

    All in all, Labourey and Kawaguchi have built a thriving commercial entity on top of Jenkins, while at the same time keeping Jenkins as a healthy, growing open source project. Kudos to them for a job well done.

    That is not to say that there aren’t competitors to Jenkins. There are some open source projects and some commercial projects that claim to do CI better and faster. Of course many of them benefit from having Jenkins already there as a template and just improving incrementally on this already there. You can see a full list of Jenkins competitors at: https://www.g2crowd.com/products/jenkins/competitors/alternatives

    While some of these competitors offer an improvement here or there, by and large Jenkins is still the standard against which they are measured. With CloudBees guiding and managing Jenkins, it will probably stay that way for the foreseeable future.

  • Why Continuous Integration Doesn’t Work

    Why Continuous Integration Doesn’t Work

    Continuous integration is easy. Download Jenkins, install, create a job, click the button, and get a nice email saying that your build is broken (I assume your build is automated). Then, fix broken tests (I assume you have tests), and get a much better looking email saying that your build is clean. Then, tweet about it, claiming that your team is using continuous integration.

    I’ve seen it multiple times in multiple projects. The start is always bright and smooth. Problems start later when the build gets broken again and we simply don’t have time to fix it. We’re simply busy doing something else and a few broken unit tests shouldn’t be a distraction for us. After all, we all know that unit testing is not for a team working with deadlines, right?

    Wrong. Continuous integration can and must work. The latest build should always be clean. Always.

    What is Continuous Integration?

    Nowadays, software development is done in teams. We develop in feature branches and isolate changes while they are in development. Then, we merge branches into master (Git usage is assumed). After every merge, we test the entire product, executing all available unit and integration tests. This is what is called continuous integration, abbreviated as CI.

    Sometimes, some tests fail. When this happens, we say that our “build is broken”. Such a failure is a positive side effect of quality control because it raises a red flag immediately after an error gets into master. It is a well-known practice, when fixing that error becomes a top priority for its author and the entire team. The error should be fixed right after a red flag is raised by the continuous integration server.

    # Why Continuous Integration Doesn’t Work?

    CI is great, but the bigger the team (and the code base), the more often builds get broken. And, the longer it takes to fix them. I’ve seen many examples where a hard working team starts to ignore red flags, raised by Jenkins, after a few weeks or trying to keep up.

    The team simply becomes incapable of fixing all errors in time. Mostly because the business has other priorities. Product owners do not understand the importance of a “clean build” and technical leaders can’t buy time for fixing unit tests. Moreover, the code that broke them was already in master and, in most cases, has been already deployed to production and delivered to end-users. What’s the urgency of fixing some tests if business value was already delivered?

    In the end, most development teams don’t take continuous integration alerts seriously. Jenkins or Travis are just fancy tools for them that play no role in the entire development and delivery pipeline. No matter what continuous integration server says, we still deliver new features to our end-users. We’ll fix our build later. And it’s only logical.

    # What Is a Solution?

    The solution I propose is simple — prohibit anyone from merging anything into master and create scripts that anyone can call. The script will merge, test, and commit. The script will not make any exceptions. If any branch is breaking at even one unit test, the entire branch will be rejected.

    In other words, we should raise that red flag before the code gets into master. We should put the blame for broken tests on the shoulders of its author.

    Say, I’m developing a feature in my own branch. I finished the development and broke a few tests, accidentally. It happens, we all make mistakes. I can’t merge my changes into master. Git simply rejects my push, because I don’t have the appropriate permissions. All I can do is call a magic script, asking it to merge my branch. The script will try to merge, but before pushing into master, it will run all tests. And if any of them break, my branch will be rejected. My changes won’t be merged. Now it’s my responsibility — to fix them and call the script again.

    In the beginning, this approach slows down the development, because everybody has to start writing cleaner code. At the end, though, this method pays off big time.

    In my projects I automate this merging operation using www.rultor.com, a free hosted DevOps service. Its usage is explained in this article: Rultor.com, a Merging Bot. I would be glad to answer your questions, just shoot me an email.