devopsy.kinsta.cloud

Category: Features

  • DevOps as a Service at base2Services

    DevOps as a Service at base2Services

    Given DevOps’ emphasis on close collaboration, what are we to make of the idea of DevOps as a Service? Aaron Walker, cloud and integration principal at base2Services explains.

    base2Services started as a consultancy service specializing in open source software (especially middleware), but also developed software for its clients. Over time, the pendulum swung towards development, but clients also wanted the company to provide hosting services and ongoing delivery management.

    In the late 2000s, “We gravitated towards cloud because it was a natural fit… we could manage infrastructure as if it were software,” says Walker. “We were a big Agile shop, so it made sense to apply those practices to operations.”

    By 2012 base2Services realized it had built significant capabilities in this area, and so abandoned software development to concentrate on infrastructure management with a focus on ‘infrastructure as code,’ and migrating clients’ systems to the cloud and then providing ongoing management.

    “We don’t do anything that doesn’t have a cloud element,” says Walker, who explains that base2Services can help clients take advantage of the cloud even if they are still using a traditional piece of software.

    Because of the company’s history, “the DevOps thing was something we just did,” says Walker.

    base2Services’ goal is to enable its clients to move more quickly, and it does this by carrying out the tasks that a client is not yet ready for, by working to minimise any gap between its own and the client’s teams, and through skills transfer.

    The company focuses on continuous improvement, and on the development of a continuous integration and delivery environment for the client.

    base2Services’ clients come from a broad range of sectors, says Walker, but especially e-commerce as well as traditional enterprises and government departments that are moving into the cloud. The former “get the fact that they need to be in the cloud,” while “we’re bringing traditional IT departments along on the DevOps story.”

    In some cases, base2Services is brought in to challenge an internal IT team. This makes it important to show “we’re here to help and we’re not there to replicate what they already do,” he says.

    This approach has proved successful in helping other IT organizations become more agile so they can focus on serving their colleagues on the business side through much shorter cycles.

    Physical proximity of development and operations staff tends to be a characteristic of DevOps, and that would seem to work against base2Services’ strategy of providing the Ops part as a service.

    Not so — “We’re very hands on,” says Walker. When working remotely, base2Services staff take advantage of ChatOps and also attend customer standups via Skype or similar technologies.

    “It all depends on the customer and its maturity,” he explains. More maturity means more daily interaction; less maturity means base2Services people are more likely to physically attend the customer’s premises on the days they are required during each sprint. “We’re pretty flexible… we’ll work with customers how they want to work.”

    base2Services customers include fashion e-tailers Cotton On and The Iconic, utility service provider Service Stream, and state government agency Emergency Management Victoria (EMV).

    EMV is “a really great story of DevOps working to deliver a platform very quickly,” says Walker, as the new system had to be deployed between bushfire seasons. Multiple partners were involved in the project, with base2Services providing full automation for the environment, plus ongoing management including the deployment of new features and capabilities. “It wouldn’t have been able to happen so quickly under a normal government structure — they would have still been at the requirements specification stage.”

    The challenge with new customers is that they need to understand that DevOps is a process, Walker explains. “I cringe when I see job ads for a DevOps engineer,” he says. “DevOps is not a person or a set of tools, it’s a culture. You have to facilitate the culture and the processes when introducing it to an organisation, and use DevOps as a guiding principle to get people to work effectively together.”

    Sometimes the customer does not have the right internal structures for the successful adoption of DevOps, in which case base2Services assists with the transformation.

    “The biggest problem we find is that while teams naturally self-organize in smaller organizations, enterprises and government departments tend to be quite rigidly structured with defined roles and responsibilities — but by its nature, DevOps has to be cross-functional.” This leads to conflicts, because people become unsure of their place in the team and exactly what they are responsible for.

    The answer is education, combined with gradually introducing the various stakeholders to DevOps at an early stage of the project, he says. “You want them to take shared ownership, and the way to do that is to carve out one piece at a time rather than trying to change the whole organisation at once. This approach builds energy and enthusiasm, which encourages other stakeholders to engage with the process.”

    It is also important to avoid being over-prescriptive: “Pick one aspect — perhaps automation of a certain task — and adopt that as a starting point.” That way, even large and traditionally structured organizations can benefit from DevOps.

    For some people, “it becomes religious… ‘You have to follow all these practices or it’s not DevOps’,” but that is not constructive according to Walker. His advice is to avoid getting stuck in the details, and instead make a start and commit to constant improvement. Seeing through small but meaningful changes will deliver value, he says, because the introduction of DevOps is one of many situations where it does not make sense to try to boil the ocean.

  • On beginning software delivery acceleration

    On beginning software delivery acceleration

    Time and time again, we hear of companies achieving rapid acceleration with DevOps. Companies are touting success with the metric of deploys/day, sharing new baselines of 10, 50 or even 100 deploys/day. In more mature organizations the likes of LinkedIn, Netflix, Etsy, Facebook and others, this number is a startling 1,000+ number. But, what does this even mean?

    Unpacking this number is often one IT leaders stumble on, and is the greatest source of pushback when talking about rapid acceleration. How stable is the software being delivered? How many of these deployments are successful? What constitutes a deployment?

    These questions and more often end up creating analysis paralysis, and nothing changes. In fact, the most common response starts with, “There is no way we could implement something like that at my company because…” We know this to be false. Companies who have traditionally delivered in a waterfall fashion with a single, bloated deploy per month or quarter are successfully transitioning to rapid deployment posting similar baseline numbers as other teams.

    How then can you be successful in beginning software delivery? There is no single silver bullet or magic wand, but there is a fairly prescriptive methodology for achieving our goal of rapid delivery. Let’s dive in and take a look on how to get started.

    Ground Rules

    Let’s set the stage with some key principles that will light the path ahead. They are:

    • Embrace failure
    • Blamelessly analyze failures1
    • Quickly repair failures when understood
    • Always be iterating

    In a nutshell, the goal should be to Deploy Fast, Fail Fast, Learn Fast, and Improve Fast. This is a never-ending task, and it is important to consider it as such. Too many times, a project is spun up to start working on these tasks, and inevitably that project ends. Unless continual improvement becomes a core tenant of delivery, any efforts risk falling apart at the seams over time.

    With these rules in mind, let’s continue.

    How is software developed?

    The first stop is incidentally not within the delivery organization, but rather the development organization. This is where the empathy that is so often discussed alongside DevOps will come in; we first need to ensure the developers who are delivering features are equipped for success.

    How is software developed? What does it really mean to get into the flow of production? Ask yourself questions like: how is code introduced into the source tree? What does acceptance of that code look like? (For example: are there code formatting guidelines, are tests needed, etc).

    Once this is understood, the key is the gate that exists to advance code to the next stage of delivery. This is where bottlenecks often exist, and often have valid historic cases for existing. Usually, this is to try and improve quality by introducing checkpoints, but these checkpoints are often human driven, and will invariably contain defects of their own.

    It is important to make the distinction around delivery of code or delivery of features versus the delivery of the software itself. Both are important as illustrated in the next section, but two sides of the same coin that are often separated duties. In the context of the entire system, the need is the same: identify opportunities to reduce friction by either removing steps from process (non-value add unnecessary, vestiges of old days) or introducing automation to move things along.

    These conversations are always enlightening, as it will become pretty apparent where the slowdowns occur after some initial conversations. Keep those brain juices flowing, and let’s continue.

    Keep in mind:

    • Who can write software? Everyone?
    • What is the barrierto entry associated with learning how to write software?
    • How difficult is it to demonstrate working code to a colleague?
    • How difficult is it for a colleague to run your demonstration code?
    • Who can deploy the software?
    • What is the barrier to entry associated with learning how to ship the software?
    • Where is work being assigned?
    • Where are feature discussions happening?

    Kicking things into high gear

    By this point, it is possible that you have a laundry list of things you have already identified as areas to improve. But, where do you focus? The default tendency is to clean up shop first, and then go work with external teams to try and integrate better. However, that is actually the least ideal thing to do, because it ends up creating silos of isolation. Possibly against your better judgment, plan on integrating first.

    Your first steps should be focused on two objectives. The first of which is to embed yourself into the development work stream… get work items from development via their existing process and find yourself an advocate. This advocate will be your “first follower,” and will help you understand what changes are working and which are not, and will provide valuable and honest feedback that may not come from other members not because they don’t care, but as their energy is focused on other things. It is almost a necessity to start small.

    The point of empathy that comes up most times DevOps is mentioned will come into major play here. No matter what efforts that you make to try to improve things, any friction introduced into the life of a developer will dramatically affect your success in any real change. For example: It may be advantageous for you to change the underlying ticketing platform for you all to communicate better, but if you disrupt the existing delivery pipeline, the odds of making new friends may go down a bit.

    Work with your advocate to firm up how development reports issue to your team and vice versa, and run it through the gauntlet. Do not forget to apply ground rules… Deploy Fast, Fail Fast, Learn Fast, and Improve Fast. Fix easy process holes quickly. Do be mindful of the amount of change that is being introduced at any given time, don’t be afraid to make changes, but also be consistent and measured with this change. This will set the appropriate tone for sustained and long-term process improvement.

    Likewise, do a real project via this first process. One early victory can be to ensure that developers have the ability to run any version of code (git checkout, major version, doesn’t matter) with minimal effort.

    This allows me to run other colleagues’ code to test out fixes, confirm bugs, or whatnot. By having the ability to run arbitrary code, collaboration speeds up closest to the code, and the number of problems that bubble up to later phases of deployment are cleaned up much earlier. This may mean staging environments; this may mean local development environments. But, solving the smaller task of how developers can run code becomes a test bed for how a production delivery pipeline may behave.

    When you find failures in both process and tooling, it is essential to be blameless while analyzing and working through failures. Early on, there will be many, and it is essential to set tone early on. John Allspaw talks about this in his article when he discusses incentives.

    Because an engineer who thinks they’re going to be reprimanded are disincentivized to give the details necessary to get an understanding of the mechanism, pathology, and operation of the failure. This lack of understanding of how the accident occurred all but guarantees that it will repeat. If not with the original engineer, another one in the future. – Allspaw

    Another often-overlooked concern is how these solutions are implemented. Repair of these processes cannot be driven by a single person or in a top-down fashion. Rather, all members of the team must be a part of owning the solution. Time and encouragement will go a tremendous length.

    Finally, deploy. The entire point of this project is to deploy code. The work done up to this point largely will focus on the interaction between Development and Operations, but assuredly there is work to be done on both sides. Until you are able to deploy daily without any breakage, that should be your target. Every day, attempt a code deploy regardless if you know the outcome already. Pick one of two things to fix daily: a failure in the process, or a manual step. Repeat this until you can deploy each day with a minimal number of commands (best case: single command).

    The Path Ahead is Dangerous, Take These!

    As you go along this path, there will be two themes that appear to come up time and time again. The first has to do with communication. Processes to deploy may get faster or more streamlined over time, but there are always ways that communication roadblocks cause challenges for the best of teams.

    The second always has to do with tool disparity. Once engineers get wind that tools need to be implemented, all people and process concerns become secondary. This typically becomes a quagmire, as these tools need to actually participate in the communication streams within the organization to facilitate software delivery.

    To this end, there are two tools that are very beneficial to keep in mind as you go down this path: ChatOps2 and Workflow3,4. Both of these topics deserve entire sections dedicated to their own right, but in a nutshell, here are the two things that should whet your palette.

    ChatOps brings the context of the work you are already doing into the conversations you are already having. – James Fryman

    Human readable, machine scriptable language for declaring workflow is not just convenience. It’s a necessity for dynamic creation, uploading and updating workflows on a live system, and for treating “infrastructure-as-code.” – Dmitri Zimine

    Just remember: it’s not about one tool. These tools are a means to an end. The people and process have to be part of this conversation. ChatOps enables the communication and collaboration that creates and drives the development. Workflow enables collaboration around infrastructure process. Both of these will become essential.

    Closing

    To succeed, we as technologists need to know what’s going on in our organization, how technology efforts are helping delivery, and how the technology being deployed is being consumed and by whom. Without this understanding, we’re just throwing tools at a problem and often wind up complicating matters. These tools can jam things up or become shelf ware because nobody is ready to implement them. And in the rapidly moving, lean businesses of today, there is no time for breakdowns.

    Do you know how your teams deliver and collaborate around software? Do you have a tight feedback loop established between your teams and those you provide services to? Are you moving fast, and fixing even faster?

    How teams deliver software today is quickly being recognized as a market differentiator. This journey is neither easy nor quick, but will enable your organization to be prepared for the next phase of rapid growth in the industry.

    ###

    1https://codeascraft.com/2012/05/22/blameless-postmortems/

    2https://speakerdeck.com/jfryman/chatops-technology-and-philosophy

    3http://stackstorm.com/2015/04/10/the-return-of-workflows

    4https://www.slideshare.net/slideshow/embed_code/key/cHCIlAlU4Xb1Lw

    About the Author/James Fryman

    ph-james-2015-05-12James Fryman is an Information Technologist who builds, designs, curates, and evangelizes automation in all layers of the IT stack. Over the last decade, he has held roles in Information Technology that include the domains of Information Security, Service Delivery, IT Operations, Development, and IT Management. He has learned through these experiences the importance of automation in all facets of Information Technology to accelerate delivery, reduce human errors throughout an application lifecycle.

    James most recently worked at GitHub assisting in the development and curation of systems scaling within the Operations group. He is also a frequent speaker on the topic of automation at conferences throughout the world.

    James holds a BS in Information Technology from the University of Phoenix.   Twitter: http://www.twitter.com/jfryman

  • Container Usage Survey

    Container Usage Survey

    Is there any topic in IT hotter than containers today? Hard to make a case that there is. Whether it is Docker or Rocket, LHX or something else, it seems everyone is talking about them. But what are people actually doing with them? devopsy.kinsta.cloud and ClusterHQ are conducting a survey on actual and planned container usage. The results will be published just before DockerCon next month.

    We need your help though to make the survey as insightful as possible. The more responses we have the better. So we are offering a $50 dollar Amazon gift card to one out of every 50 people who take the survey. The survey is only about 20 questions and shouldn’t take more than a few minutes to complete.  We don’t ask for contact info unless you want to be eligible for the gift card giveaway.  Either way taking the survey is a big help to us, ClusterHQ and everyone else as well.

    The link to take the survey is:https://www.surveymonkey.com/s/3TZ9LDC

    One of the key areas of inquiry is how many people are actually using containers in production environment. How many have already used containers versus how many are planning to do so? What about data management in containers? These areas and more are explored in the survey.  It should make for some excellent research.

    Thanks for your help in making this a great survey!

  • Release Management, a catalyst for DevOps

    Release Management, a catalyst for DevOps

    We have discussed in my last post “DEVOPS and Continuous Change” on the benefits of doing shorter and swifter runs instead of managing a few ‘Dinosaur’ releases. We have also touched upon the way to go about multiple releases by identifying component /functional dependencies without impacting functionality and the expected savings of time, resources and cost, reaching the goal of rapid delivery of software.

    It’s common to find a formal Release Management (RM) team in large organization, with a lot more of medium and some small ones following suite. We will not touch upon the benefits of having a core RM team helping plan and manage the complex journey of change from the handover of code by the Application teams, packaging and deploying across testing environments by Operations, validating testing signoffs, risk analysis, impact assessment and ultimately ensuring a successful Production deployment.

    Rather, let’s delve deeper to understand how this Confounding or the Third variable called RM fits into DEVOPS. In my opinion a very important actor which neither belongs to DEV nor OPS!

    I would call Release Management a catalyst in true sense, which enables change without itself undergoing such change, a neutral team dedicated in managing the change by brokering a formal hand-off between DEV and OPS.

    RM teams traditionally have been the most underrated, with the DEV and OPS hogging the credit of every successful run to production. They converge the business understanding of new features or fixing a functionality break with the Development team’s new version of code release or the Operations expectation of infrastructure upgrades which will also provide an enhanced user experience advantageous to business and development.

    So, back to my question, where does RM team fits in the cozy DEVOPS selfie?

    Wouldn’t it be incorrect to ignore RM from the DEVOPS equation?

    I would take the liberty of stating that the DEVOPS movement in an organization should be owned, driven and determined by RM, adding Change/Configuration and Release management expertise to the existing cross-functional mix of the DEVOPS team.

    RM should be driving this collaborative approach to rapid delivery, bringing together the right mix of people from both sides and ensuring efficient use of resources, empowered and with executive oversight supporting hard decisions.

    They would be able to add value in this rapid delivery process, by not just tasking them with managing a release cycle but empowering and enhancing their capabilities. RM team under the DEVOPS banner should also be involved in these broad points below:

    • Prioritizing the book of work, creating trade-off between Business needs and Development delivery.
    • Understanding complex business requirements along with adhering to geographical and geopolitical needs and rules.
    • Distributed Set of applications, architectural complexities, interdependent and independent components and dependencies,
    • Complete awareness of the infrastructure landscape, scalability, capacity planning, migrations.
    • Release planning, calendar management, process adherence, true agile sprints, optimal resource utilization, Contingency or Rollback strategy planning.
    • Post deployment/release lessons learnt, retrospection, streamline or integrate processes after every run.
    • Ensuring consistent message is sent across that the DEV and OPS are not adversaries rather, partners in this collaborative approach of software delivery.
    • Management reporting along with any risks assessed or highlights deficiencies.

    This take is based on the many DEVOPS discussions which I have come across where there is no mention of RM or is being assumed as ‘part of DEV and works with OPS’.

    My disagreement on this thought process is not about to which group it essentially belongs or will be part of, but in assigning an inconsequential role to RM in the whole release journey when in fact they should be the drivers!

    Thank you for your time.

    Visit my blog for earlier posts http://askhurram.com/

    Read On, Live On.

     

  • DevOps Leadership Series 2015

    DevOps Leadership Series 2015

    We recently hosted the 2nd annual “DevOps: Wine-ing, Not Whining” event in San Francisco to bring together a number of thought leaders and leading practitioners in the DevOps field.  We traded stories from the trenches, formed new insights, and offered visions of the future.

    This week, I’ll be sharing videos captured from the event that highlight key themes from our conversations. We call it the DevOps Leadership Series.

    Screen Shot 2015-05-15 at 11.19.59 AM

    You’ll get to hear:

    In the next few weeks, I’ll be on the west coast at the Velocity Conference and on the east coast covering DevOps Days DC.  I’ll be capturing more insight from DevOps thought leaders at these events.  If you are planning to attend one of these events, let’s capture your DevOps perspective for the series.  To participate, please leave a comment below or catch me on @weekstweets.

    Today, we’ll start the series with Gene Kim on important themes for DevOps in 2015:

    [youtube http://www.youtube.com/watch?v=0IXrnhcRNUk]

     

    PART 2 of the series is now available at: 

    https://devopsy.kinsta.cloud/2015/05/19/devops-leadership-series-software-supply-chains/

  • Survey: Fed gov cloud success hinges on DevOps

    Survey: Fed gov cloud success hinges on DevOps

    A new survey out last week by MeriTalk and Accenture Federal Services shows that the federal government is on the cusp of finally putting its money where its mouth is with regard to cloud and that DevOps will play a big role in the transformation. In fact the government’s cloud success may hinge on DevOps.

    The survey looked at perceptions of federal IT managers familiar with their agencies’ cloud progress. Many of them report difficulty in gaining cloud momentum, in spite of the fact that 66 percent believe that their agency needs to move IT services to the cloud faster to meet mission and constituent needs. Fewer than half of Federal agencies have made process or policy changes to facilitate successful transitions to the cloud, only 30 percent have made necessary cultural changes and just 28 percent have made organizational changes. What’s more just 12 percent of Feds report their departments have all the tools they need for a cloud transformation.

    However, 57 percent believe DevOps holds the key to help them succeed in the cloud and 63 percent say DevOps will help them speed up application delivery and migration. As things stand, only one in ten respondents believe that developers and administrators are highly collaborative in the Federal setting and only 5 percent say their agency has fully deployed DevOps patterns.

    “A DevOps model brings software engineering, quality assurance, and IT operations together as an integrated team,” stated Tim Hoechst, Accenture Federal Services’ chief technology officer (CTO).  “Federal agencies can start up new projects more quickly, improve resource utilization, and reduce downtime – ultimately driving the innovation needed for real change.”

    Respondents are hopeful of that vision, with 60 percent reporting they see DevOps in their agency’s future. Approximately one-third say they’ve already started adopting the model or plan to do so within a year.

    But it’s not clear whether this is pie-in-the-sky hopefulness for a silver bullet or a real commitment to DevOps. A big red flag from the survey was that some of the same barriers to Federal cloud adoption are those that typically stand in the way of DevOps transformations. For example 40 percent of Feds said fear of change and inflexible practices have stood in the way of their cloud efforts.

    Over half of respondents say they will need to train current personnel to implement DevOps, and 40 percent say DevOps success depends on incentivizing changes in culture to make it happen. Meanwhile 41 percent believe they’ll need to establish a new vision for the future to make DevOps possible.

  • Netuitive’s Transformation to SaaS Delivery, Powered by Its New DevOps Shop, Part Two

    Netuitive’s Transformation to SaaS Delivery, Powered by Its New DevOps Shop, Part Two

    In part one, devopsy.kinsta.cloud presented Netuitive’s DevOps transformation, which helped the company to automate its product development processes. Now in part two, we take a look at that development process as it stands today.

    Netuitive Development & Technical Debt Initiatives

    “We are using a Feature Branch Workflow model with Git to isolate our work while we develop and test it,” says Brian Spindler, VP of Engineering, Netuitive. As soon as all teams and team members are in agreement, Netuitive merges the feature with the master branch and deploys it to the internal pre-release staging environment for further testing and verification.

    “In the old world, we released software twice a year,” says Spindler. After using the Scrum development process religiously and exclusively to take the software to beta, Netuitive drove the pre-release software through the testing ringer prior to releasing the packaged final product.

    “Now we release on a weekly basis,” says Spindler. The development team shoulders the unit, integration, and acceptance testing while Netuitive reaches forward to increasingly add features and polish its architecture.

    Netuitive is continually bringing the increasing technical debt into the black side of the development ledger, so to speak. Netuitive offers two examples of being technically in the red. The first is in the area of unfinished business. Incomplete or hacked up implementations typically happen when the team is developing outside the core technology stack, is cutting corners for speed, has a poor understanding of the software requirements in a given area, has to rush testing, or does not create complete and appropriate documentation, according to Spindler.

    The second example circles library / dependency updates, where falling behind can lead to numerous interrupts and blocks for features / bugs / requests that are at the top of the queue, Spindler explains. “Often times in either of these two sample areas, a theme will emerge and that can be a good indicator that it is time to start paying down the technical debt in that area,” says Spindler.

    Netuitive puts that debt under a microscope where through transparency and visibility, it can review the bugs on a regular recurring basis and, as soon as an engineer catches something that is offensive to the software, he can create a ticket in JIRA to put it on the backlog to address, according to Spindler.

    “However, it doesn’t always make sense to ticket something if it is not actionable,” says Spindler. In such cases, Netuitive will catch those instances of technical debt in the Technical Roadmap Idea Board that it maintains, which captures three verticals: the development environment, the technical debt, and the architectural / structural vision, according to Spindler. “Any team member can contribute to this Board. To keep everything on it in perspective, we hold retrospectives regularly,” says Spindler.

    Changes, Experiences

    “It was a positive experience seeing how we handle bug fixes differently after switching to a SaaS model,” says Spindler. Now we understand bugs that we fix very well and we easily duplicate them for isolation and to excise them from the software. That’s because the developer has full access and freedom to debug the same system that he reports on. This eliminates the tedium of setting up systems and testing and repairing for edge case scenarios, according to Spindler.

    “We were also positively surprised at how quickly something can go from concept to production,” says Spindler. To make that possible, there is an opportunity cost in that development has to be willing to sacrifice, to try something while risking having to toss something aside. “In the SaaS world, it’s all about trade-offs, fast vs. refactoring, pay off that technical debt or put out feature XYZ. When you have a team like ours, everyone’s time is critical, and we work hard at prioritizing and balancing that,” Spindler says.

    To maintain the balance and improve the team’s precision and good judgement over time, Netuitive follows Kaizen, which is a continuous improvement methodology. “As the team grows and changes, so will our priorities and the way we work and build the system. We’ve got to be open to that and willing to embrace it,” says Spindler. Regular retrospectives and transparency help to make that possible.

  • Surprise! Broad Agreement on the Definition of DevOps

    Surprise! Broad Agreement on the Definition of DevOps

    For the past few years, the conventional wisdom has been that DevOps has no agreed upon definition. Instead, everyone has their own interpretation. However, the conventional wisdom is wrong.

    While everyone supplies slightly different words, the same working understanding of DevOps is nearly universal. Our understanding of DevOps goes back to a handful of common touch-points. There is the “CAMS” (Culture, Automation, Measurement and Sharing) acronym popularized by John Willis and Damon Edwards. CAMS has been extended repeatedly – Eveline Oehrlich of Forrester is now discussing CALMSS. Gene Kim offers his “3 Ways” and Patrick Debois has codified “4 Areas”. Allspaw and Hammond had 10 deploys a day, and Etsy made technologists fascinated by a virtual crafts bazaar.

    Analysts, authors, industry, and community leaders all agree. The ideas are big and mutually supportive. In short:

    • DevOps exists to help the business win
    • The scope is broad, but centered on IT 
    • The foundations are found in Agile and Lean.
    • Culture is very important
    • Feedback is fuel for innovation
    • Automation helps

    Why: To help the business win

    The most popular DevOps book today is The Phoenix Project, with a subtitle “A Novel about IT, DevOps and Helping Your Business Win”. DevOps is not a navel gazing IT methodology for making IT better for IT. It is focused on how stuff involving the technologists can deliver better results for the business.

    As Patrick Debois writes in his excellent post,”DevOps Areas – Codifying devops practices“:

    As rightly pointed out by Damon Edwards, devops is not about a technology, devops is about a business problem. The theory of Constraints tells us to optimize the whole and not the individual ‘silos’. For me that whole is the business to customer problem, or in lean speaks, the whole value chain.

    At IBM, our definition of DevOps calls out enabling the business to “seize market opportunities”.

    Today, technology is critically important to how a wide variety of companies compete. Retailers who neglected e-commerce have gone out of business. Logistics companies like UPS find a competitive edge by creating software to shave miles off delivery routes. In the automotive space, Ford is at tech conferences recruiting companies to write apps for its cars and Tesla’s first fix for their flammable battery problem was an over the air software update.

    DevOps is a response to what the IT is experiencing – from a back-office concern that needed to “keep the lights on” to key part in how the business competes every day.

    Who: A Broad Scope, beyond development and operations.

    Yes, DevOps is a portmanteau that brings development and operations together. Yes, the classic DevOps concern is the Wall of Confusion between development and operations. However, with the focus on business value, the scope must include everyone who is involved in taking an idea through IT to business value.

    In recent years, there’s been something of a cottage industry of community leaders and analysts writing, “No, no it really should be Dev_____Ops” where the blank is filled in by their own specialty. Examples: DevQAOps, DevOpsSec, DevSecOps, BizDevOps and of course Bussdevtestqanetsecnetops.

    When a Finance department must approve the provisioning of each new test environment, slowing the delivery of new applications by weeks or months, there is a DevOps problem that leaves narrow confines of development and operations. It leaves IT. Patrick’s quote above talks about optimizing the whole value chain. In that post he describes DevOps as being about “collaboration, optimization across the whole organization. Even beyond IT (HR, Finance…) and company borders (Suppliers)”.

    Based On: Agile and Lean

    In answering why he didn’t support a manifesto for DevOps, Patrick Debois points people back to the Agile manifesto and asks them to reread it more broadly. Agile’s shift towards a faster delivery of software, and focus on the end goals make it a natural spiritual ancestor for DevOps. However, from the perspective of Ops teams, Agile represented the barbarians at the gates. Agile generally took root in development side of the house, enabling it to create something the business wanted faster. But with operations still moving at a pace suited to waterfall, pressures grew and the need for something like DevOps became clear.

    Where much of DevOps energy seems to come from its Agile roots, the intellectual side of the movement is more rooted in Lean. It seems no accident that many of the Phoenix Project’s lessons are taught away from the cube farm and instead from the factory floor. While we can intuit that faster feedback is better, and the “continuous” in CI and CD is convenient because it lets us map feedback to fewer changes, it is in the Lean literature where we encounter Little’s Law. It provides the queue theory mathematics supporting the idea that smaller batch sizes supports shorter cycle time. And so Reinertsen’s “The Principals of Product Development Flow: 2nd Generation Lean Production Development” has become popular in DevOps circles and is well cited by Jez Humble in his new Lean Enterprise book.

    Culture: Collaboration and Experimentation

    The Agile and Lean roots show through in the DevOps culture as well. Lean and Agile are both focused on people first, systems second and heavily cross-train or work in cross-functional teams. They also emphasize continual improvement (kaizen) through approaches like retrospectives.

    In DevOps circles, collaboration is preached as an antidote to thinking in siloes which has resulted in antagonism between development and operations. For the systems thinking required to deliver for the business, teamwork is required. Developers make better decisions when they take advantage of operations knowledge (Patrick’s Area 3) and operations does better when application knowledge is embedded with them (Area 4).

    The other cultural emphasis is a push towards experimentation, constant learning and improvement. This is Gene Kim’s third way of DevOps. The Lean Startup ideas of Eric Reis permeate the DevOps world. Intuit has made great strides in showing how these approaches work in established companies by running experiments in the midst of tax season resulting in improved business outcomes.

    These cultural trends come together in things like blameless postmortems discussed by Etsy and Jake Loomis in Web Operations. When things go wrong, organizations need to understand why and take corrective action. In troubled corporate cultures, the messenger reporting the problem might be shot, and the hunt to assign blame for the problem will begin. Healthy organizations look to learn from failure and try to improve the systems that allowed (or guided) people to make a series of mistakes that lead to the failure.

    Feedback: Yes. Lots please!

    Experimentation, adaptation and learning tend to do better when there is ample feedback. Otherwise, there is too little to learn from. Developers tend to be more productive as well with rapid feedback. Their thirst to know whether the latest change diminished quality led to the practice of continuous integration and the tools that support CI and continuous delivery. That CD ends up facilitating faster delivery of value to the customer is something of a happy accident.

    Amplifying feedback loops is Gene’s “second way“. The “CAMS” concepts of measurement and sharing point to feedback. You gather lots of information, and make sure people see it so they have an opportunity to learn from it. And when Elisabeth Hendrickson presents at DevOps Enterprise 2014, as Cloud Foundry’s Director of Quality Engineering at Pivotal, she describes her job as “to see to the care and feeding of the feedback cycles“.

    As expected with a collaborative culture with ample sharing, feedback must cross the traditional silo lines. That means the awesome production monitoring that had lived in Ops land, is radiated back to development. This is “Area 2” in Patrick Debois codification.

    Automation: Of course.

    The “A” in CAMS is automation. Patrick’s “Area 1” points to continuous delivery techniques, well described by Dave Farley and Jez Humble in their book. A Lean initiative looking at IT will find waste in wait times, errors, and more and move towards automation as a natural result. Feedback craves automated deployment of changes, and automated tests or production monitoring that validates them. Automation is what makes a lot of DevOps possible. It doesn’t make cooperation between development and operations happen, but tensions are reduced when Ops gets out of the business of arguing about the SLA they have with Dev around server provisioning and instead provide Dev a button to press.

    DevOps isn’t about automation. Automation is just the natural result of DevOps.

    Additional Resources not cited above

    DevOps for Dummies – by Sanjeev Sharma

    8 Best Practices for DevOps – Ashok Reddy’s overview of the Forrester Report

     

    About the Author/ Eric Minick

    ericMinickEric Minick is a Technical Evangelist. He joined IBM through its acquisition of UrbanCode. He has spent the last ten years helping organizations – large and small – adopt continuous integration, delivery and now –DevOps. He is a frequent speaker on the topic, and co-authored Application Release and Deployment For Dummies.

    Prior to consulting, Eric was a developer, test automation engineer and support engineer, and has contributed to multiple generations of UrbanCode products.  Follow him on Twitter at @ericminick.

  • Netuitive’s Transformation to SaaS Delivery, Powered by DevOps, Part 1

    Netuitive’s Transformation to SaaS Delivery, Powered by DevOps, Part 1

    “Netuitive offers an adaptive monitoring solution for cloud infrastructure, applications, and services, powered by a patented self-learning correlation engine,” says Brian Spindler, VP of Engineering, Netuitive. The engine uses multivariate regression analysis to calculate correlation coefficients between metrics based on time-series data collected, leveraging the significance of certain relationships between metrics variables across the data.

    The Netuitive technology establishes a baseline of what metrics and their relationships have been historically. Netuitive policies enable development and operations teams to see anomalies that present with significance above the historical baseline and related to AWS EC2, Linux OS, MySQL, and other systems and technologies.

    “We have launched a SaaS version of our product wherein we now deliver code to production on a weekly basis,” says Spindler. This increase in deployment speed is thanks to the company’s new DevOps processes. Here is the first part of Netuitive’s two-part story.

    Before Moving to a DevOps Shop

    Before moving to a DevOps software development approach including continuous delivery, Netuitive used a Scrum framework together with Jenkins and Ant products and scripts the company developed in-house to develop Netuitive software and the patented self-learning correlation engine within. “We recognized the value of automation early on—using simple shell scripts and our own Web API we were able to orchestrate quite a lot,” says Spindler.

    Still, these methods presented challenges. The extensive planning process, including conjuring up development time estimates that Scrum requires for Sprints were challenging. In addition to and despite all that planning, numerous interrupts occurred surrounding testing and deployment issues that stemmed from software complexity and integration management. “These interrupts slowed us down and impacted our burn rate,” says Spindler. (The burn rate calculates the speed of the business spend above and beyond income.)

    Mountains of bug fixes were also a concern. With an exorbitant amount of time spent fixing bugs that would only present themselves in rare edge cases, Netuitive was putting too much effort into builds while getting too little in improvement or new features out of the process. “While we sought a solution for feedback loop issues, it became clear that this would require a more agile approach. We looked to DevOps for inspiration, which led us to migrate from Scrum to Kanban,” says Spindler. Kanban is an illustrative process for managing targeted production goals, including what to develop when and how far to extend that development.

    As enterprises adopting DevOps often mention, the cultural change necessary to move to DevOps approaches was the most challenging. In particular, getting everyone on board with automated testing as a proof of software validity was tricky. Internal teams such as QA and developers had to transform their existing relationship. “For years, these groups were adversaries; now I am asking them to be allies. This required the developers to take on the responsibility of writing tests while QA validates and ensures the tests are sufficiently broad,” says Spindler.

    Goals, Results, & Hazards: Method to Madness?

    Netuitive had specific outcomes in mind from switching to a more complete DevOps shop. Quality and productivity improvements were high on the list. In fact, the two would have to be mutually inclusive and close knit to ensure both. To accomplish this, Netuitive deploys on a weekly basis, using the full Atlassian suite for collaborations and issue tracking along with a combination of Bamboo, Gradle, and Salt Stack to automate the configuration, manage the infrastructure, and manage deployment operations. This orchestration helps quality and productivity to dance in harmony.

    The results are more than aesthetically pleasing. “We are increasing the velocity of deployments through continuous process improvements,” says Spindler. By deconstructing the Netuitive product and dividing it up into independent service modules and components, the company can move overall progress along more quickly by improving each individual component separately and simultaneously, making faster progress gradually than they would by waiting to finalize a single, complete product update.

    Netuitive experienced three fully distinct bumps in the road or ‘hazards’ on the way to morphing itself into a DevOps shop. “There were issues related to automating testing, changing the organizational workflows to accommodate automation, and deconstructing the product into granular enough services that we could deploy independently,” says Spindler. These were all related to the desire to automate the product production as a whole. Netuitive works continually to refine the necessary balance between layering on additional tests across those granular services to raise the bar on quality and excising some of those testing layers in order to increase deployment velocity.

    Tune next time when devopsy.kinsta.cloud reproduces the relative chronological Netuitive DevOps development process from release to release along with differentiations, benefits, and the payoffs that the company did not expect.

  • Step-By-Step: How DataArt Processes Application Backlogs, Converting Legacy Apps with DevOps

    Step-By-Step: How DataArt Processes Application Backlogs, Converting Legacy Apps with DevOps

    DataArt designs and custom builds software for industries such as travel & hospitality, media, IoT, financial services, and healthcare. “We build new products and services, improving existing solutions, re-engineering legacy systems, and providing expertise in technological and industrial domains,” says Anton Krasikov, Project Manager, DataArt.

    How DataArt Uses DevOps

    DataArt quickly delivers high quality software in support of operations performance. DataArt’s DevOps process enlists version control systems like Git and build systems such as Ant, Maven, and Gradle. “Almost every project relies on using these build systems inside CI servers like Jenkins, TeamCity, or TFS,” says Krasikov.

    DataArt’s development process also requires a lot of unit and integration testing using frameworks like TestNG and Junit. “There are also projects where we prefer TDD or BDD approaches, with the use of specific frameworks such as Selenium for UI testing,” says Krasikov. By using Puppet, Vagrant, and Chef, DataArt can swiftly deploy the development environment in the CD pipeline. DataArt applies these select DevOps development tools using methodologies, which could include Kanban or Scrum (Scrum was most frequently mentioned by DataArt for this story).

    Previously, DataArt relied on manual processes or scripting tools for build systems, source code, and unit tests. “It took a lot of management effort to tie everything including development, quality assurance, and operations together,” says Krasikov.

    Challenges to Adding Backlogged Applications to Those You Develop Using DevOps Approaches

    In the instance where DataArt returns to work on existing systems that include some legacy applications, it is possible to use agile development in maintaining these as they are, but excessive software dependencies could make it hard to align the software releases / release cycles.

    “Legacy software may not have good test coverage or a proper build system, so we usually take steps in a DevOps direction, if not to achieve continuous delivery, at least to implement proper continuous integration,” says Krasikov.

    There are challenges to adding backlogged apps to a DevOps development process. “These kinds of non-DevOps applications usually suffer from several issues, or in worst case scenarios from all of them,” says Krasikov. Backlogged apps waiting in a pre-DevOps state and hungry for transformation may have any or all of the following symptoms, according to Krasikov:

    *many dependencies between system modules
    *no automated environments
    *no continuous integration
    *no test automation
    *no build system or inefficient usage

    Transforming an Application, Step-By-Step

    To add apps to DevOps approaches, says Krasikov, start by analyzing the existing source code, system dependencies, and system environments. The steps that follow come from an example with a recent Java project that DataArt did.

    A) Select a source code control system such as Git.

    B) Do refactoring and perform unit tests. “These go hand-in-hand, so the main business logic was covered with tests. It does not make sense to rush to 100% test coverage, but we tested the main bits. This also allowed us to reduce system complexity and decouple system components,” says Krasikov.

    C) Build system – “The original application used huge and confusing Ant scripts to produce deployable packages. After we completed the refactoring, we split the application into modules with Maven as the build and dependency management system,” says Krasikov.

    D) Integration tests – “We introduced these as a separate test automation suite to ensure that application modules work correctly as a whole,” says Krasikov.

    E) Continuous Integration – DataArt set up a Jenkins server for multiple build jobs, building with each commit and targeting deployments to their intended environments with overnight integration testing. “CI allowed us to identify non-working builds early to save time on investigations,” says Krasikov.

    F) Environment setup – Finally, DataArt implemented infrastructure as code. The new development environment uses Puppet with Vagrant to check out source code, spin up a VM box / template, and deploy fast, according to Krasikov.

    DataArt runs some steps in parallel for additional time savings, for example moving to virtualized provisioned environments along with source code migration, according to Krasikov.

    DevOps Helps DataArt Speed Projects to Their Customers’ Delight

    By increasingly converting backlogged apps to DevOps development approaches, the enterprise benefits by achieving project goals and delivering a working software iteration with each pass through the DevOps processes. “The high level of automation reduces the risk of human errors and helps us to fail early in the development cycle. With continuous delivery, it is much easier for our clients to see the immediate impact of their ideas,” says Krasikov.