When building DevOps or continuous delivery (CD) practices relying on Docker, you can learn a great deal from others. Reference architectures reveal what choices people made, what practices they put in place and how they connected the dots.
We at Sonatype pulled together a set of 21 reference architectures for folks building CD and DevOps practices using Docker. Why? DevOps professionals numbering 3,500 recently attended our webinar on, “Continuous Integration using Docker,” discussing just one reference architecture example. We recognized then there was a strong interest in the community to learn more. That’s why we assembled 20 more reference architectures into this collection.
A picture speaks a thousand words. Beyond the images, we also share the details of each DevOps/CD Docker reference architecture. In the bottom right corner of each slide, we share the original blog or presentation source for the image. When one of the images piques your interest, you can then dive deep into the source. Cool, huh?
Think of this set of reference architectures as open source. The slides are free to view, download and distribute. Share with others if you find them helpful.
Help Others
Oh, and if you have a Docker reference architecture we should add to the deck, please let us know. We would love to add it to our deck for others to learn from. Just like the others we included, you will be sure to get full attribution for your work.
For those of you not using Docker, we did publish a set of DevOps and continuous delivery reference architectures last year on devopsy.kinsta.cloud. With more than 86,000 views and 3,300 downloads, many people in the community have found it helpful.
SAN JOSE, Calif.–(BUSINESS WIRE)–The Jenkins community, the community of practitioners using open source Jenkins, and lead organizing sponsor CloudBees, Inc., the enterprise Jenkins company and continuous delivery (CD) leader, today announced a full program of add-on training, workshops and the ability to take a Jenkins certification exam for free at the upcoming Jenkins World conference, to be held September 13 – 15 in Santa Clara, CA.
Jenkins Certification Training and Free Certification Exam: September 12-13
This course, delivered by experts from |opensourcearchitect| and Nebulaworks, will prepare attendees for the Jenkins certification exam. It will cover all exam topics and there will also be a series of labs on several subjects addressed throughout the course. Attendees will:
Learn from Jenkins experts: Both |opensourcearchitect| and Nebulaworks are certified to deliver Jenkins training and teach courses about Jenkins on an ongoing basis.
Prepare for certification: Get ready for the Certified Jenkins Engineer (CJE) and Certified CloudBees Jenkins Platform Engineer (CCJPE) exams.
Take the training, then stick around for the conference: Jenkins World offers 50 sessions across a variety of topics and skill levels.
Take a free certification exam: Offered at convenient times throughout the conference, and free, as long as you sign up for it at the time of conference registration.
Fundamentals of Jenkins Pipeline and Docker Training: September 12
This course is an introduction to the Pipeline plugin, used in combination with Docker. Following an overview of Pipeline and Docker, participants will:
Learn about the need for Pipeline: Understand key use cases, structure/syntax and execution control.
See how to prepare Docker environments: Get exposure to key Docker tools and learn how to create a deployment lifecycle with Pipeline and Docker.
Gain hands-on experience: Watch and learn, live, as you work on two actual projects with Pipeline and Docker experts.
Workshops: September 13
The DevOps 2.0 Toolkit Workshop
This workshop will cover the entire microservices development lifecycle, from architecture design through to production and beyond.
Let’s Build a Jenkins Pipeline!
Jenkins Pipeline is a game-changing way to write automation jobs with Jenkins. Come learn how.
Preparing for Jenkins Certification
Assess your readiness for the Jenkins certification exam.
Introduction to Plugin Development
This workshop will introduce developers to the Jenkins plugin ecosystem and terminology, as well as identify the developer resources available.
CD and DevOps Maturity Workshop for Managers
In this workshop, we will review the definition of DevOps, its relationship to continuous delivery (CD) and explore a simple, lightweight model for measuring organizational DevOps maturity.
Thanks to the leading technology vendors who are supporting the Jenkins community by sponsoring Jenkins World. Sponsors include: CloudBees, the premier sponsor and lead organizer, joined by: BlazeMeter, Red Hat, Codenvy, cPrime, JFrog, Sauce Labs, SOASTA, Sonatype, Black Duck Software, Cisco, GerritForge, IncrediBuild, MacStadium, Perforce, XebiaLabs, Infostretch, Mobile Labs, OpenMake Software, pCloudy, Perfecto Mobile, Rainforest QA, Site24x7, Undo Software and ZeroStack.
Training Partners include: |opensourcearchitect| and Nebulaworks.
Media sponsors include: devopsy.kinsta.cloud, DZone, PAN Communications, TechTarget SearchSoftwareQuality and TechTarget TheServerSide.com.
About Jenkins and the Jenkins Community
Jenkins is the leading open source automation server supported by a large and growing community of developers, testers, designers and other people interested in continuous integration, continuous delivery and other modern software delivery practices. Built on the Java Virtual Machine (JVM), it provides more than 1,200 plugins that extend Jenkins to automate with practically any technology software delivery teams use.
The Jenkins community advocates the use of Jenkins to the global development community, serving as a central source for tutorials, forums and other helpful resources for Jenkins users of all experience levels. By recognizing the numerous contributors to the Jenkins project, the project creates and fosters a community-powered infrastructure for maintaining and further developing Jenkins. You can follow the Jenkins community on Twitter (@JenkinsCI).
About CloudBees
CloudBees (www.cloudbees.com), the Enterprise Jenkins Company, is the continuous delivery (CD) leader. CloudBees provides solutions that enable DevOps teams to respond rapidly to the software delivery needs of the business. Building on the strength of Jenkins, the world’s most popular open source continuous delivery hub and ecosystem, the CloudBees Jenkins Platform provides a wide range of CD solutions that meet the unique security, scalability and manageability needs of enterprises. The CloudBees Jenkins Platform supports many of the world’s largest and most business-critical deployments. Follow CloudBees onTwitter, Facebook,LinkedIn and Google+.
This is part three of the series, Tales of DevOps Discovery. To catch up, Read part one and part two.
In preparing for a DevOps transformation a critical first step is to perform upfront discovery, often asking even the most fundamental, seemingly obvious, questions of stakeholders. The major portion of such discovery should be done in a group setting. Asking stakeholders these fundamental questions in a group setting can provide key insights into differing perspectives and even differing objectives regarding a planned DevOps implementation. Sharing these different perspectives during the discovery phase sets the stage for mutual empathy and enables stakeholders to identify and align, an important step to aligning on shared objectives. In this article we will outline common questions asked, share real-world answers and offer guidance on interpreting and applying the results.
Discovery Question: What do you want to achieve with this transformation/DevOps?
One of the first things to consider is, “Why are we doing this?” as this establishes the goals and objectives. Accordingly, I usually begin by asking the stakeholders during discovery what they expect to achieve through the transformation to DevOps. The answers I receive to this question often differ distinctly, depending on the role of of the person answering. Understandably, they are heavily influenced by the unique perspective of that role.
Answers:
CIO: “We want to increase our ability to “scale-up” delivery as we shift to a primarily SaaS (software-as-a-service) model and see a rapid increase in business. These means we have to increase developer productivity, and respond to customer requests faster, all while reducing our number of Severity #1 issues.”
Operations:
“We want to reduce technical debt, to minimize firefighting and increase our ability to adopt modern best practices such as Infrastructure-as-code. We have to be able to keep the lights on while delivering applications to production.”
Development:
“We would like to minimize customer-specific one-off work, which is resulting in painful complexity in development, QA (quality assurance) and deployment, and is an overall source of inefficiencies. We would like to be able to spend more time improving user experience, performance and increasing valuable functionality. But, instead, we are chasing bugs and reworking misunderstood requirements.”
Guidance:
Not surprisingly, executives often respond with a focus on business-oriented goals such as increased productivity and customer response, while Operations focuses on stability and minimizing firefights and Development wants to innovate. But within these differences we see some commonality that can be harnessed, such as productivity. For example, one point of commonality is productivity:
CIO: “We have to increase developer productivity.”
Operations: “… minimize firefighting and increase our ability to adopt modern best practices …”
Development: ” … spend more time improving user experience, performance and increasing valuable functionality.”
Another point of commonality in the CIO, Development and Operations answers is the focus on quality:
CIO: “ … reducing our number of Severity #1 issues.”
Operations: “We want to reduce technical debt … keep the lights on …”
Development: “… instead, we are chasing bugs … ”
These points of commonality can be used to establish common goals with cross-functional buy-in, such as:
Our effort will result in increased productivity and minimize overtime and firefights (across teams)
Our efforts will result in increased application quality and uptime
They then can be used not just as driving principles that inform decisions such as process or tool choices, but also to establish key performance indicators (KPI), which will be used to quantifiably measure success of the transformation. Example KPIs may be:
Reduction in cycle time, from story acceptance to user acceptance (or release)
Number of Severity 1, or Severity 2 production issues per release
Defects per build
Defects per release
Hours worked by Development? Operations?
Discovery Question: Do you practice agile today? If so, how do you define done for a story or sprint?
Continuous delivery (CD) and lean practices are foundational to a successful, sustainable DevOps transformation (though not required to meet the some definitions of DevOps.) If we look at CD and lean practices as a pipeline through which we need to manage a continuous flow of changes represented by work, then it is important to consider what work is sent into the pipeline, how the work is initiated and how the changes are managed. Accordingly, it is important during discovery to ask customers high-level questions about development methodologies used and overall practices for definition, planning and coding.
The answers given vary between different levels of, “Yes, we are agile,” “No, we are waterfall,” and, “We practice both.” Most Most often, organizations’ answers describe adherence to a form of agile with room for improvement.
Answers:
“We practice agile and/or scrum. We work in two-week iterations, planning the work for a sprint at the end of the prior sprint.
We consider a story done once the code has successfully been committed and merged to a team branch, and is successfully built. We consider a sprint complete when all stories for that sprint are committed and successfully built.
“Roughly every three months we integrate code from team branches to trunk and perform a mix of automated and manual regression and acceptance tests. Once the build passes QA we manually deploy to preproduction environment, validate and schedule a manual deployment to production.”
Guidance:
This is a classic story of agile adoption in traditionally waterfall organizations. The organization adopts agile within Development without involving other key functional teams such as QA or Operations. Usually, this is because it simply is easier. There are a number of obstacles to overcome in extending agile practices across organizational, technical and cultural boundaries, so the obstacles are avoided by limited the agile adoption to these boundaries.
This is a case of confining the adoption to “agile upstream” while practicing waterfall or legacy “downstream.” Studies show that less than 33 percent of organizations have adopted “agile upstream” and exercise agile or iterative development practices for the planning, coding and team builds, yet only 13 percent of organizations have adopted agile downstream, and follow practices such as continuous integration (CI), CD, automated provisioning testing and deployment.
The negative effects of limiting agile adoption to agile upstream are multifold, but can be summarized as, “You never really know when you are done-done.” As a result, you cannot accurately measure progress, introduce deferred risks or disrupt the development plan to address issues found during the downstream portion of the process, such as poorly understood requirements surfacing during acceptance testing or runtime issues discovered during deployment to a production-like environment or, in the worst case, issues found in production by the end user.
A seminal book on DevOps is, “The Phoenix Project,” by Gene Kim, which presents three ways of DevOps thinking. The first way, “Systems Thinking,” addresses to this gap between agile upstream and downstream, Dev and Ops, business and customer by taking a systems view across the traditional boundaries and establishing “flow” of the system through the boundaries.
Discovery Question: Do you practice CI today? What types of validation do you perform?
As CD is foundational to DevOps, CI is foundational to CD. CI can be scene seen as the connection point for agile upstream to agile downstream, and the pump that fills the CD pipe. It is both the practice most organizations say they follow and the practice that most often is implemented incorrectly. So when surveying a customer’s current state during discovery, it is important to know whether CI is currently part of the picture.
Answers:
“Yes, we practice continuous integration. Each team has a Jenkins CI server. Developers commit their changes multiple times per week. Jenkins then builds the changes as part of a nightly build.”
“We do some unit testing, but we do not track code coverage. If the nightly build fails we may continue to commit, but we usually fix the issues within a few days.”
Guidance:
This is a common answer and an example of how CI can be improperly implemented. CI is more than having a shared Jenkins CI server and running scheduled builds. CI can be defined as:
“A a software development practice where members of a team integrate their work frequently, usually each person integrates at least daily – leading to multiple integrations per day. Each integration is verified by an automated build (including test) to detect integration errors as quickly as possible.” — Martin Fowler
The benefit of practicing CI includes identifying integration and quality issues close to the source of the issue, thus reducing the time to correct and enabling higher-quality software to be developed more rapidly.
Essential elements of properly implemented CI are :
Integrating changes frequently, ideally multiple times per day or even per change.
Performing consistent validation of quality through component level testing, code scans, etc.
Always maintaining a working build, suspending development when a build fails, and not resuming development until the issues is corrected or reverted.
Teams that allow a broken build for “a few days,” deferring commit or builds to just “multiple time per week” or that do not validate consistently through testing practices are not practicing CI and will not yield the benefits of CI.
After a successful spring run, devopsy.kinsta.cloud, where the world meets DevOps, along with premier sponsors, CloudBees, the enterprise Jenkins Company and IBM, today announced that their DevOps Connect: CD Summit/Jenkins Days roadshow will continue this fall with seven dates in the United States and Europe. The call for speakers to appear at these events is now open. Today we are announcing the shows will be coming to Atlanta, Denver, Austin, Seattle, Los Angeles, Berlin, and Amsterdam.
I began my employment at devopsy.kinsta.cloud as a web admin and social media manager. Shortly thereafter, I became marketing and media manager. I had decent education surrounding web scripting (HTML/PHP/CSS/Photoshop), but none about DevOps and software engineering. I attended the spring events and had a front-row seat to the summits and what they had to offer. The keynotes offered a plethora of information surrounding Jenkins, BlueMix and how to undertake a DevOps transformation at the corporate level. We were honored to have Kohsuke Kawaguchi, CTO of CloudBees and founder of Jenkins, offer his unique perspective on the growth of Jenkins, joining CloudBees, and the development of recently released Jenkins 2.0.
Our other two keynotes featured individuals we were lucky to have. Gary Gruver, who led the transformation at HP and Macy’s, outlined what he walked into, how he changed the company, and what happened when we walked away. He gave a clear-cut story about the transformation he embarked on when joining those companies. The third keynote came from Mustafa Kapidia of IBM. Mustafa was such a captivating speaker who really knew how to gauge his audience’s knowledge surrounding DevOps and continuous delivery.
The breakout sessions in the afternoon following lunch offered a bit of something for everyone. There were three separate tracks: technical, management and Jenkins. The technical track was just that: a track devoted to the more technical-oriented crowd such as engineers and developers. The management track was devoted more to those looking to undergo transformations, incorporate DevOps at their workplace, and manage DevOps teams. The Jenkins track focused solely on information that was connected to utilizing Jenkins in the workplace.
In addition to the summit itself, CD Summit and Jenkins Days feature a day dedicated to workshops on DevOps, IBM BlueMix and Jenkins. These were a hit from the get-go, pulling in people of all levels, from developers to executives wishing to broaden their knowledge. The fall will feature the workshop day as well.
I met a lot of people during my time at these events, cementing the fact it is a great place to network and learn about different companies and how they are doing devops. Prior to attending these events I only knew a small chunk about DevOps/CD/CI based on the articles we publish on devopsy.kinsta.cloud and from my interaction with professionals in the industry. By attending the events I expanded my knowledge, greatly improving my capacity to understand what occurs from the junior level to the executive level at a company utilizing DevOps practices and principles.
According to the press release, “These events are designed for Enterprise Managers and Buyers along with Developers, Engineers, Sys Admins, and Architects. With three main tracks focused on IT managers, technical engineers, and Jenkins practitioners, attendees will get access to the best minds in CD and DevOps. The presenters are experienced developers, build managers, QA, DevOps practitioners, IT managers/executives, architects and IT operations who are luminaries within the DevOps community.” I would recommend attending this event for anyone who fits this description, and for anyone outside the realm of this description who wishes to learn more about DevOps or continuous delivery.
Today we are opening up the Call for Papers for the fall events. Anyone who would like to submit their entry can follow the respective links below for each city:
As anyone who has ever been set up on blind dates by friends can attest, finding the right match in today’s world is a tricky feat.
There’s a lot of evidence suggesting that the frantic pace of life makes such matchmaking more difficult than ever. That said, when the right set of complimentary traits exist and there’s a willingness to find common ground, things can work out.
Spike Jonzian-notions of artificial intelligence aside, certain technological partnerships are increasingly being perceived as “a perfect marriage” within the context of DevOps automation.
Register today for the upcoming webcast: “Using APM to Turbocharge Continuous Integration”
These technological matrimonies closely support the underlying tenets of DevOps methodology—collaboration, experimentation and increased velocity. In fact, it’s easy to wonder why no one ever conjoined them before. But we were using traditional waterfall development and operations and so couldn’t conceive of such a thing. On the other hand, emerging availability of tooling supporting new integrated workflows represents a significant catalyst.
Finding the Perfect Fit
One such DevOps confederation getting a lot of attention is the matchmaking of continuous integration (CI) and application performance management (APM) technologies. Tying together these once-disparate systems makes a lot of sense, despite the fact that few tools would seemingly appeal to more (traditionally) polarized constituencies.
Continuous integration has been a staple of the development community for many years. It emerged as teams evolved the practice of long build times into more frequent check-ins to deliver code and look for potential errors more quickly. Today, with the rise of CI tools such as the Cloudbees Jenkins platform, some development teams are touting tens of thousands of builds per day.
On the other end of the spectrum, squarely on the turf of IT operations, APM has been evolving rapidly over the past few years as organizations have sought to improve use experience and applications reliability via dedicated monitoring. Enabling this ability to identify and address emerging issues has resulted in APM becoming the fastest-growing segment in the IT ops management market, according to Gartner.
DevOps matchmaking factors in when you bring these tools together to create an automated workflow that spans their respective use cases. A cross-functional team can leverage APM-level monitoring and KPIs within the domain of CI and testing to identify potential performance obstacles introduced in new code. Doing so then rather than waiting to respond later in the applications life cycle benefits everyone.
Reaping the Benefits
For example, say an e-commerce applications developer has implemented a new inventory search functionality—something considered a key differentiator by business management. The developer knows this feature could rely heavily on some back-end database system and wants to ensure it’s tested properly, even as the code is being introduced.
Integrating APM and CI allows the entire team to monitor the process and test various conditions related to the update, tracking multiple builds and looking out for related performance issues. Such integration also allows for centralized tracking of other key attributes such as build number, status and build date, to further ease management.
Most importantly, by tying these two critical aspects of the applications delivery life cycle together, the collaboration and feedback loops required to truly embrace DevOps methodology become ever more valuable as these core Dev and Ops functions become intrinsically linked.
If this type of DevOps advancement gets your heart pumping, you’re not alone. Later this month, we’ll host a webcast here on devopsy.kinsta.cloud featuring experts from CA Technologies and Cloudbees. They’ll be discussing the benefits of matchmaking with APM + CI integration. They’ll also detail real-world benefits organizations leveraging integration of our respective technologies (CA APM and Cloudbees Jenkins) enjoy.
We obviously feel that this is a DevOps tooling marriage that will mature, change and stand the test of time. We’d love to garner your questions and feedback on the topic.
Which is all good, because thankfully there’s no such thing as Tinder for DevOps … yet.
I haven’t written much about some of the great content and sessions that have been held in our DevOps Connect: CD Summit/Jenkins Roadshow. We have held them in Dallas, Chicago and Boston. Each event was different, but all featured some tip top speakers. One of the “regulars” at our events has been Kohsuke Kawaguchi or KK as the Jenkins world calls him. KK is the CTO of CloudBees and the creator of Jenkins. He is also one of the more intelligent people I have ever had the pleasure to meet. On top of this his humble nature only enhances his overall stature. I wanted to post a short interview I did with KK at the Dallas DevOps Connect: CD Summit/Jenkins Days. We hope to have his full keynote available on our DevOps TV YouTube channel soon as well.
Below is the the video of my interview with KK and below that is a transcript to read along as well. Also a quick reminder next week (May 23 & 24 and May 25) we are in London and Stockholm continuing this fantastic road show. Next fall we will be in 8 or 9 more cities as well. Check out DevOpsConnect.com for dates.
Alan Shimel: Hi, this is Alan Shimel, editor in chief at DevOps Connect CD Summit Jenkins Days in Dallas. Happy to be joined by KK, the founder of Jenkins and CTO of CloudBees. KK, thank you.
KK Kawaguchi: Oh, thank you very much for having me.
Alan Shimel: Thank you – well, it’s our pleasure to have you, but you are just coming out of the Jenkins track that you delivered, which was a very technical track on Jenkins and your keynote this morning was on continuous delivery pipeline and we spoke about Jenkins 2.0 as well as local Jenkins as a service and Jenkins certification. And what was the last one where we give you the whole package that we’re going to not certify, but what was the word?
KK Kawaguchi: So you call that the CloudBees Jenkins platform, private label solution basically.
Alan Shimel: Absolutely. So, for our listeners who weren’t lucky enough to be here, can you – I’m not going to ask you to do the whole 45 minutes, but can you give us a – just in five minutes, sort of the highlights of each, of what we spoke about?
KK Kawaguchi: Yeah, so Jenkins version two, like, we are trying to move to dial in three key areas. The one is the continuous delivery through the pipeline of code, so we want people to be able to define this choreography of orchestration, AKA, the continuous delivery pipeline as a series of text files. And then they can put that into version control system and monitor them well. So that’s one of the sort of key features in Jenkins 2.0.
The second one is this more curated out of the box experience. So we want new people who are starting Jenkins today or tomorrow to start with this 80 percent, like a richer set of functionality that represents the 80 percent of the use cases so they have right out of the box, they get something useful and they can get a lot of mileage from. And then at some point, they would start leveraging this great ecosystem in Jenkins. So that’s this out of box experience is the second part.
And then the third one is some of the GUI improvements. You know, Jenkins is something people will use every day, so we wanted to make sure that we keep up with this progress in the frontend technology. So that’s something we’ve been doing. So those are the three key pillars to the new release.
Alan Shimel: And I think you said that fingers crossed, we’re looking at the Jenkins 2.0 or CloudBees 2.0 in the end of this month?
KK Kawaguchi: Yeah, so the Jenkins 2.0, we are – we’ve been working on this for a good while now and we are at the final home stretch. Today is actually the release candidate day.
Alan Shimel: Really?
KK Kawaguchi: Yeah, and then in two weeks, if nothing bad shows up, and that’s the part where I’m crossing fingers, nothing bad shows up, in two weeks, we are shipping the 2.0 GA.
Alan Shimel: So as you mentioned in your keynote, as someone who started this as a little project for your three, four man team inside, I imagine there’s a tremendous amount of price in seeing what this has become and –
KK Kawaguchi: Yeah. Yeah. Little did I know that the itch that I was scratching was shared by so many other people.
Alan Shimel: Yep, hundreds of thousands, really.
KK Kawaguchi: Yeah, exactly.
Alan Shimel: What was the number, 100,000 active installs, 400,000 computers?
KK Kawaguchi: Yes, that’s right. I think the exact number I believe I think is 120,000 or something, but there’s a massive number of people using Jenkins to, I guess to help move their – to help deliver software faster and I’m really proud of being a part of that.
Alan Shimel: Fantastic. KK, one last question. Many of our listeners and readers out there, they’ve heard of Jenkins. They’ve heard of continuous delivery. They may not know how to get started with it. For someone just looking to get started, what’s the best way?
KK Kawaguchi: Right. So if you are really new to the whole, you know, whole CICD, then I think that the very easiest place you can start is, well, you know, the install Jenkins, it’s really easy to just really get it installed or you have it installed and so you can start one up and then you put the – like an automated build up on every commit.
So, everyone already has a build screen because that’s going to run that every time someone commits a change, even something that simple, which only takes like 15 minutes, it’s going to pay a lot of dividend.
And then once you get this idea across to your team that well, a little bit of automation helps, then you can like incrementally go a little bit by little bit over time. So next step might be running tests. Maybe that could take a lot of, you know, multiple phases. And then that’s how I start on the, like, how Jenkins spreads inside some microsystems is really where that came from. That’s how Jenkins got into so many places and that’s how people are doing it, one thing at a time, sort of a little bit over a little bit.
Alan Shimel: Excellent. KK, thank you so much for your time. Thank you for sponsoring, CloudBees for sponsoring our CD Summits and we look forward to seeing you in future CD Summits as well as Jenkins 2.0.
KK Kawaguchi: Yeah, thank you very much and me too.
Alan Shimel: Thank you. This is Alan Shimel for devopsy.kinsta.cloud.
New major release accelerates software delivery with focus on pipeline-as-code, software pipeline stage views and usability essentials for new and existing Jenkins users
SAN JOSE, Calif.–(BUSINESS WIRE)–The Jenkins project, comprised of the community of practitioners using open source Jenkins, today announced the availability of Jenkins 2.0. The first major release after 10 years and 655 weekly releases, Jenkins 2.0 addresses the demands of teams challenged with matching the pace of continuous software delivery while ensuring quality and providing visibility across the full software delivery process. Updates to core Jenkins capabilities include first-class support for continuous delivery (CD) pipelines, clear visibility of delivery stages and enhancements to help users get started more quickly.
“These new features will ultimately enable us to implement robust continuous delivery process automation and deliver higher quality software faster than before.”
Tweet this“As Jenkins user adoption continues to grow into millions of users, the community has responded with core updates to assure continued, long term success,” said Kohsuke Kawaguchi, creator of Jenkins and CloudBees CTO. “The Jenkins 2.0 release helps new and existing users implement continuous delivery with the most flexible way to model, orchestrate and visualize the entire delivery pipeline.”
Jenkins 2.0 Updates
Continuous Delivery Pipeline Support:
The Pipeline plugins provide orchestration of all stages of the software delivery process, with enhanced visibility and feedback. Jenkins 2.0 has several updates to support continuous delivery pipelines.
Pipeline-as-Code enables teams to quickly create and collaborate on simple to advanced CD pipelines using code.
Stage View enables teams tovisualize, collaborate and act on the application delivery value stream in real time (see Figure 1, above).
Pipeline Durability ensures that long-running pipelines withstand outages by resuming at the point of interruption, significantly improving productivity.
Multibranch provides automatic, zero-configuration of CD pipelines for new branches and pull-requests.
Usability Enhancements:
Jenkins 2.0 includes several enhancements that make it easier to use, and enable users to quickly be more effective.
Get started more quickly with community-recommended plugins: A new quick start UI provides solution-oriented plugin bundles. These are the most commonly-used plugins, selected from the more than 1,000 available plugins, to enable a fast start for new installations (see Figure 2, above).
Better user experience: Jenkins 2.0 includes enhanced configuration pages and other UI updates to make configuring and using Jenkins more efficient.
Upgrade confidently: Today millions of users depend on Jenkins, making backwards compatibility a primary concern. Jenkins 2.0 is designed as a drop-in replacement for Jenkins 1.x.
New community website: In response to the rapidly growing Jenkins community, the new Jenkins website makes it easier for new users to get started with Jenkins, encourages user contribution and increases visibility into Jenkins events.
“By providing pipeline-as-code capabilities and visualization of software pipeline status within Jenkins, Jenkins 2.0 allows me to define both simple and complex pipelines,” said Shashikant Jagtap, DevOps and automation engineer, Jenkins user and lead for the London-based Jenkins Area Meetup. “These new features will ultimately enable us to implement robust continuous delivery process automation and deliver higher quality software faster than before.”
Additional Resources:
Attend the TweetChat about Jenkins 2.0 today – April 26, 1pm EDT:
Log on to Twitter and search for #jenkins2
About Jenkins and the Jenkins Community
Jenkins is the leading open source automation server supported by a large and growing community of developers, testers, designers and other people interested in continuous integration, continuous delivery and other modern software delivery practices. Built on the Java Virtual Machine (JVM), it provides more than 1,000 plugins that extend Jenkins to automate with practically any technology software delivery teams use.
The Jenkins community advocates the use of Jenkins to the global development community, serving as a central source for tutorials, forums and other helpful resources for Jenkins users of all experience-levels. By recognizing the numerous contributors to the Jenkins project, the Jenkins project creates and fosters a community-powered infrastructure for maintaining and further developing Jenkins. You can follow the Jenkins community on Twitter (@JenkinsCI).
DevOpsQA NJ recently held its April Meetup on Automation with Jenkins and BDD, with Kishore Bhatia from CloudBees and Tom Bartolucci from Billtrust as speakers. The venue, food and refreshments were provided by Audible at a state-of-the-art, vast office space in historical city of Newark.
After fun networking over pizza, wine and beer, we kicked off with Tom Bartolucci’s presentation on BDD Automation framework using Codeception. Tom has been in web software development for over 13 years. He has built applications in Java, .NET, PHP, Python and Javascript for clients in publishing and mobile video. Currently, he is a VP of Development at Billtrust – a SaaS-based payment cycle management company. His team of 13 web developers works on delivering six of the Billtrust SaaS products and uses Codeception, a PHP-based BDD framework to test.
According to Tom (and we all agree), the idea is not to find bugs but prevent them. His team at Billtrust chose Codeception because it allows tests to be written in PHP, which is what its development teams use; it also supports Selenium WebDriver, PhantomJS and BrowserStack; and has hooks for various tools and platforms and build-in reports. As a result, it had a low barrier of entry and got Codeception’s team off the ground quickly. Doing cross-browser testing allowed them to find IE bugs, which they previously missed due to testing primarily in Firefox. Codeception allows testers to create BDD-style readable tests, similar to Cucumber spec but written as code. The easy syntax enabled the QA team to write code that they could read and understand. And developers are able to add custom modules because the framework is developed in PHP, which they are familiar with. Codeception has a variety of uses including to write acceptance tests, unit tests and API tests.
Tom also described how Billtrust integrated Codeception with Jenkins to execute tests and publish reports. He then proceeded with a live demo of the framework. At the end of the session there were a lot of good questions on Vagrant integration, parallelization, failed test rerun and differences between Codeception and Cucumber.
Next, Kishore Bhatia from Cloudbees gave a talk on Jenkins pipelines and Gerrit Code reviews. Kishore Bhatia works forCloudBees, building custom frameworks with open source software and helping customers solve engineering problems around continuous delivery and DevOps at scale. Kishore is a Jenkins expert with development experience in FinTech enterprise software and Cloud platforms. Couple of months back, Kishore and Issac gave a presentation on Jenkins Workflow, demonstrating how workflow removes bottleneck of connecting multiple pipelines together.
This time, the focus was on a very valid use case: a large codebase, many teams contributing to the code. Teams should not collaborate on emails. They should get notifications and post-commit validations done by the tool that they use. In this case it’s Gerrit—Google open source project, offers its own implementation of Git. You just want the job to magically run behind the scenes doing the code reviews for you. The primary challenges in this scenario are that you want to fail-fast—report CI results as soon as any steps fail and make sure there are no orphaned builds.
Kishore then demonstrated the steps involved in implementing this use case in Jenkins including adding pull request validation to the build step, allowing developers to get build status directly in the VCS tool only coming back to Jenkins if something fails. Currently there is a limitation with posting each status to GitHub. Key things that need to be set up—Gerrit infrastructure, Jenkins installation, workflow that Jenkins will refer to. While it might look complex at first it becomes easier once you start using the workflow, with such things as Groovy script helper. Jenkinsfile will define the flow of what should happen in dev, QA, staging and production. Kishore then gave a great demo of the flow and answered questions from the audience which covered parallel commits and test data management.
At the end of the sessions, we did a raffle drawing where three lucky winners got prizes sponsored by Audible, including an Amazon Echo. Looking forward to our next session.
It seems IT practitioners haven’t realized yet the scale of digital disruption. Mired in the old IT thinking, they firmly believe that sophisticated tools are all businesses need to survive the ongoing digital disruption waves. Most of them warn, “Digital transformation is changing the way we do business, the way we deliver IT services must change, businesses need to adapt the way they work.” The problem is, there’s nothing new in what they offer to help businesses work the “right” way.
Indeed, the same old IT thinking’s practices are still applied with the illusion that deployment automation and configuration management tools on their own will make businesses agile, responsive to market opportunities, and therefore in line with the digital economy’s competitive challenges. That belief misses the point for two reasons: First, it’s an incorrect understanding of what business agility is, and second, it’s a misunderstanding of the global competitive environment’s transformation.
Turning your IT organization lean and agile is the competitive differentiator your business lines desperately have been calling for. Authentic DevOps implementation is the way to go. That’s the price to pay to keep up with the pace imposed by new leading-edge companies such as Google, Apple, Uber and the like.
The changing competitive environment forces businesses to focus on innovation, market responsiveness and timely revenue
A digital economy threatens the existence of all businesses. Yes, you read it right, their existence is at stake. And that fact is underestimated by IT vendors and consulting firms.
As illustrated, we’re shifting from business environments where established businesses (incumbents) control a large percentage of market segments to competitive contexts where, attracted by the amazing digital opportunities and the adoption of cloud which significantly reduces infrastructure expenses, a growing number of startups disrupt entire industries:
The proliferation of startups created by highly qualified unemployed seniors and creative entrepreneurs, coupled with the propagation of highly innovative digital services (e.g., Internet of Things [IoT] applications) substituting traditional services, have been disrupting the vast majority of industries.
Several examples substantiate that fact: the taxi industry shaken by the emergence of technology-driven companies such as Uber, the banking industry facing new breed of competitors including PayPal, Google and Apple; and the healthcare industry increasingly disrupted by companies such as GlowCap.
To grow and prosper in the digital era, your scale, brand equity or market capitalization aren’t enough anymore. To succeed, your business lines must be on top of the digital economy’s competitive standards: competitive innovation (the ability to continuously improve customer experience), market responsiveness (the ability to deliver added-value services to the right market segments at the right moment) and timely revenue (the ability to continuously increase customer value).
The digital economy has taken IT to an inflection point—deploying sophisticated tools isn’t enough; being lean and agile is the way to go
The obligation for business lines to focus on innovation, market responsiveness and timely revenue is unprecedented for techies; it forces them to think beyond tools and address issues as unusual as organizational and operational effectiveness.
Indeed, innovation, market responsiveness and timely revenue assume greater cross-functional collaboration, IT focus on business priorities, organizational flexibility and accelerated time to market. Make no mistake, you’ll never achieve them by only deploying tools, but by also making your IT operating model flexible through proactively eliminating the organizational and operational dysfunctions that prevent cross-functional collaboration, fast problem-solving and decision-making.
That’s what is meant by making the business agile.
The new IT thinking is summarized in that simple arithmetic: Digital Competitiveness = Lean and Agile Operating Model + Effective and Efficient Tools. Without the lean and agile operating model, innovation and timely revenue are wishful thinking. Without effective and efficient tools, innovation and market responsiveness are wishful thinking as well.
DevOps is the IT asset businesses need to succeed in the digital economy
Contrary to a now (unfortunately) established belief, DevOps isn’t a technology; Chef, Puppet, Jenkins, OpsWorks, alike aren’t DevOps, they’re only pieces of it. It’s important to be clear about it.
As I’ve been implementing it, DevOps is a system of principles, practices, roles and responsibilities, governance structure and tools purposely arranged to establish cross-functional collaboration, enable accelerated applications delivery and, in turn, make the business responsive to market opportunities:
As illustrated, DevOps as a business capability is made up of three components: the Rules of the Game, the IT Operating Model and the Continuous Delivery Pipeline (CDP). Let’s explore them.
DevOps Rules of the Game create the conditions for lean and agile IT environments
Rules of the Game are a fundamental dimension of DevOps: They enable the establishment of lean and agile environments. I refer to them as the DevOps non-technological piece.
In practical terms, they’re a set of principles, practices, roles and responsibilities and tools institutionalized to ensure accelerated applications delivery.
They serve as catalysts to specific organizational and operational effective practices. Let’s take the principle, “Facilitate with system thinking (involvement of all relevant stakeholders)”—it forces application development stakeholders to adopt a collaborative mindset, which in turn breaks down organizational silos to eliminate the impediments to operational flexibility and institutionalize cross-functional collaboration.
They’re the poor cousin of today’s DevOps practices, which ignore them in favor of more technical and technology issues. That’s a terrible mistake that reduces DevOps influence vs. the digital economy’s challenges.
The agile IT Operating Model establishes the lean and agile work environment
The IT Operating Model reflects the interactions that take place across your application development value stream. In practical terms it represents the work environment derived from the application of DevOps’ rules of the game.
I always remind my clients that DevOps’ rules of the game aren’t theories, they’re the foundations of a tangible strategic capability: the lean and agile IT environment. They demand a thorough transformation of your application development work space to put together business, application development, and operations:
The lean and agile IT work space for the digital era illustrated (Credit: O.C. Tanner)
The picture illustrates what’s meant by lean and agile IT environment, it substantiates Adam Jacob’s point that “Fundamentally DevOps is about taking the behaviors and beliefs that draw us together as people, combining them with a deep understanding of our customers’ needs, and using that knowledge to ship better products to our customers.”
The IT operating model is the poor cousin of today’s DevOps practices which ignore it in favor of more technical and technology issues. Once again, it’s a terrible mistake that reduces DevOps influence vs. the digital economy’s challenges.
Make no mistake, technology matters, the CDP helps to meet digital technologies challenges
The importance of the rules of the game and the resulting IT operating model don’t suggest at all that technology doesn’t matter. Make no mistake about it, the CDP is essential; I refer to it as the DevOps technological piece. Through deployment automation and configuration management tools, it significantly accelerates application development, testing, resources provisioning and deployment.
However, one thing to be aware of is, digital economy raises technological challenges that transcend speedy delivery issues. They include:
The importance of IoT
The pivotal role of predictive analytics
The increasing importance of APIs
As a matter of fact, IoT devices generate massive data and making use of it to generate revenue requires big data technologies to integrate, automate and analyze them.
On the other hand, this ocean of IoT data has generated an API business; the idea is, to monetize their data and generate revenue, businesses must provide their customers and partners with APIs that easily give access to their systems.
Implementing the CDP as part of PaaS cloud services can help to prosper in the digital economy; it allows businesses to benefit the cheap and on-demand computing, storage and network resources provided by the cloud.
Takeaway
In hindsight, with a business-oriented perspective, the ITaaS delivery model and DevOps follow the same goal: Provide business lines with the lean and agile IT environments they need to be flexible enough and responsive to meet the competitive challenges of digital economy:
Like DevOps, the ITaaS delivery model (…) takes advantage of an agile IT operating model to help businesses succeed in the digital era. It implements consumer-provider relationships across the IT value stream to ensure organizational flexibility and operational agility.
CIOs must realize that what matters to the business is to continuously generate revenue through improving customer experience and increasing customer value. Deploying applications at the speed of light for the sake of speed doesn’t make sense from a business perspective; as long as expected business benefits aren’t achieved, speed is a useless gadget.
Implement DevOps as what it is: a three-dimension IT capability including rules of the game, IT operating model and, of course, continuous delivery pipeline.
ompany. His team of 13 web developers works on delivering six of the Billtrust SaaS products and uses Codeception, a PHP-based BDD framework to test.
hatia from Cloudbees gave a talk on Jenkins pipelines and Gerrit Code reviews. 
rimary challenges in this scenario are that you want to fail-fast—report CI results as soon as any steps fail and make sure there are no orphaned builds.
Kishore then gave a great demo of the flow and answered questions from the audience which covered parallel commits and test data management.
