devopsy.kinsta.cloud

Tag: automation

From Laboratories to DevOps Factories for Continuous Testing
The prior blog Continuous Testing and Monitoring Essentials for SDN and NFV discussed the importance and suggested practices for testing and monitoring complex systems such as software defined networks and network functions that are virtualized . Whether the product being tested is a complex network system or not, with DevOps the laboratory is no longer just a place where engineers go to test their changes. In a DevOps environment, implemented according to best practices, the laboratory is becoming an on-line operation that is reliable, usable, remotely accessible and shareable by many types of remote users that expect 24/7/365 availability.
The old “Santa’s workshop” model of development engineering (elves) hand-crafting products in specialized local laboratories and then handing off their products to operations staff (reindeer) for infrequent delivery (on Santa’s sleigh once per year) is being replaced by a fully automated, orchestrated software DevOps laboratory infrastructure akin to a modern factory staffed by robots churning out products at a dizzy pace. Elves and reindeer need to rest between deliveries and simply can’t keep up with today’s continuous delivery expectations. Santa’s “lap-top” has become an app store where users can download products anytime no matter if they have been bad or good (but be good for goodness sake!).

If a modern DevOps factory-style laboratory is a priority on your Christmas list, here are some suggestions for tool requirements you may want to include on your letter to Santa.
1. Inventory tools to record and track resources:
  All of the systems used in tests such as test tools, systems under test, and interconnection systems need to be tracked in an on-line inventory database in which each resource and the critical attributes are visible and addressable by administration staff and users of the resources. The ability to discover and import new or changed resources is essential. Without these capabilities it is impossible to keep up with the continuous changes that occur as the DevOps continuous test environment evolves to meet ongoing requirements.
2. Tools to administer users:
  With a large number of users accessing lab resources features for administration of users and user groups are essential. Large scale DevOps environments require administration of hundreds or thousands of users with varying privileges for resource access according to project priorities and schedules. The ability for lab administrators to assign privileges to individual users or user-groups is essential to control access to some resources which may be restricted for security or other reasons.
3. Ability to create, and save test configurations:
  While tests may be created to work with a variety of configurations every test assumes something about the test configuration. In a test environment implemented according to best practices the configuration of the test, or at least an abstract version of the test configuration is specified and associated with each test. Configuration data is saved in a database and may be recalled by search attribute matches when a test needs to run.
4. Ability to reserve tests and test configurations:
  With a large number of lab users in large scale DevOps environments and limited lab resources conflict resolution features are needed. The ability assign real resources to abstract configurations provide the flexibility to use lab resources more efficiently. The ability to reserve test resources and test configurations in advance according to a calendar, or immediately when an event trigger fires are important options to minimize wasted time for both users who may have limited time windows or automated processes that have non-deterministic schedules.
5. Ability to orchestrate many automated tests in parallel:
  In large scale DevOps environments the number of tests that need to run in a given short DevOps cycle may vary from a few tests to thousands depending on the complexity and nature of the change that are integrated in the cycle. The tools need to be able to scale the tests and run on as many physical and virtual test resources as are available, to minimize the total time for test.
6. Test Environment Metrics:
  The entire DevOps pipeline depends on reliable, scalable and maintainable continuous testing. Built-in metrics are essential to monitor the environment’s reliability, scalability and maintainability.
The above is a partial list of suggestions for a state-of-the art DevOps continuous testing laboratory “factories” list. At Spirent we think continuous testing laboratory “factories” are critical for successful DevOps.

You can read more about our views at Spirent.com/solutions/LabManagement

What do you think of these suggestions and do you have others that should be mentioned?

In the meantime we wish Santa, Elves and Rudolf, reindeer friends and you happy holidays and continuous happiness for the New Year!
December 23, 2015
Never Mind the Bollocks, Here’s the DevOps
What is DevOps?

“DevOps is a cultural and professional movement” –Adam Jacob

[movement (n): a group of people working together to advance their ideas.]

So, who are these people and what’s the big idea?

The people are system administrators, developers, product managers, QA technicians, security analysts, network engineers, and anyone else involved in the business of developing software.

The big idea is to increase the value your product or service delivers, more quickly with less effort.

(Having a product or service that is designed and built well isn’t inherently valuable. The value comes when your customer is able to use your product or service to solve a problem. DevOps is all about delivering value quickly and effortlessly.)

DevOps practices and technologies are evolving, but here are some of the core concepts paraphrased–be sure to read the links in the footnotes for the full explanations:

The Three Ways
- Systems Thinking
  - Learning how everything is interconnected, from concept to customer
- Amplifying Feedback Loops
  - Cutting the time between action and reaction, failure and success
- Culture of Continuous Experimentation and Learning
  - Recognizing failure is important data about how and where to improve
CAMS
- Culture
  - It’s about how people relate and work, not something you can buy
- Automation
  - Faster, less error-prone processes help amplify feedback loops
- Measurement
  - You can’t get where you want to be until you know where you are
- Sharing
  - Effective teamwork ignores organizational boundaries
ICE
- Inclusivity
  - Everyone is responsible for (and benefits from) these practices
- Complex Systems
  - Because simple systems are simple
- Empathy
  - Real collaboration requires real understanding of another’s point of view
That’s the theory from a very high level; the following Q&A provides some broad explanations of related concepts. These are actual questions I was asked in a recent discussion of DevOps.

Q: What specific skills or experience do I need before I can put “DevOps” on my resume?

A: It depends. At this point in time when recruiters and/or hiring managers list DevOps in a job posting they are almost always looking for a “DevOps Engineer” (i.e. a sysadmin who knows just enough coding to be dangerous), or a “full-stack developer” (i.e. a developer who knows just enough about system management to be dangerous). The reality is that (as of today) there is no DevOps certification, organization or even manifesto, so anyone who pretends to authority on what is and isn’t DevOps is overstating their case at best, a charlatan at worst.

Q: What is source control?

A: Source control is a repository for files that make up a modern software application during development. It maintains a historical record of changes so it’s easy to review (or rollback to) changes made to an application’s source files. The most popular source control technology today is Git, and GitHub is one of many online services used to manage Git repositories.

Q: What is Jenkins?

A: Jenkins is to task engines or “build servers” what Kleenex™ is to facial tissue. It’s currently the most widely used software product for automating various functions of software build and deployment. For example, a system might be configured such that any code change pushed to source control triggers a job in Jenkins to build, test and deploy that code.

Q: What is configuration management (e.g. Salt, Ansible, Puppet, Chef)?

A: Configuration management is a means of scripting server configurations so it’s easier to track, deploy or make configuration changes to large groups of systems.

Q: What are containers?

A: In the same way that server virtualization allows you to have many isolated “virtual machines” on a single physical machine, container technology allows you to have many isolated application environments on a single operating system. Containers have been around in Unix/Linux for a long time, they just boomed in popularity when Docker came along and made them easy to use. The next version of Microsoft Windows Server (2016) will introduce Windows containers.

Q: What are microservices?

A: Microservices are small, independent components of a software application that are built, tested and operated in isolation from the core product. Using microservices enables development to be faster and more nimble than possible when working with a large, monolithic application.

Q: What is Continuous Integration/Delivery/Deployment?

A: Continuous Integration is the practice of continuously integrating (and testing) new code with your existing source code. Continuous Delivery is the practice of automating as much of the build/test/release process as possible while still requiring a manual push to production. Continuous Deployment is the practice of fully automating build/test/release all the way into production.

Q: Are there any conferences or meetups I can attend to learn more?

A: Yes! The original DevOps conference is DevOpsDays and there are many of these events around the world annually. A popular conference with large enterprises is the DevOps Enterpise Summit. Various DevOps-related meetups (#CoffeeOps on Twitter) are springing up around the country. Check the CoffeeOps.org website for details, or search for DevOps at Meetup.com.

Q: [Do I need to understand everything above | Is this all I need] to be or do DevOps?

A: Absolutely not, on both counts. You can successfully implement a DevOps way of thinking/working by incorporating all or just some of the above, and there are many other tools and practices around delivering value more efficiently. Naturally there are extensive resources available (a few linked in the footnotes) to do a much deeper dive on all these subjects.
December 15, 2015
7 things 2016 has in store for the future of DevOps and IT
With only a couple weeks left in 2015 it’s time to start looking ahead to what next year has in store. Let’s be honest—most of us have already mentally checked out for the holidays anyway, so it’s already 2016 for all intents and purposes. I spoke with a handful of industry experts to get some thoughts on what we have to look forward to in 2016.

Let’s just jump right in:
1. Log Management will play a larger role
According to Christian Beedgen, CTO of Sumo Logic, log management will emerge as the most underrated opportunity for IT. “Thinking about log data may seem too far in the weeds for most tech industry professionals. However, using analytics to monitor, manage and gather insights from logs will be the only near-perfect way to make sense of the increasingly complex and cloud-based architectures.” Log management isn’t exactly new or cutting edge, but Beedgen believes that 2016 will see more vendors trying to move into the log management space, and more organizations trying to use log management to make sense of their big data mess.
1. Business leaders will demand better insight
On a similar note, Rick Fitz, senior VP or IT Markets for Splunk, believes that analyzing data and metrics will also play a role from a business strategy perspective. “In 2016 we will see data science demystified, moving from Ph.D. to MBA, with easier ways emerging to consume, analyze and correlate machine data. As business leaders increasingly go digital, this will result in greater “digital insight” into engagement, satisfaction, competition, effectiveness, response, revenue and other business goals.”
1. Sharing economy will transform IT
Companies like Uber and AirBnB have transformed entire market segments. The sharing economy has changed the way businesses engage customers and deliver services, and that includes IT through aspects such as cloud services, microservices, APIs and more. Fitz predicts, “Next year this ‘Uberfication of IT’ will turn into a ‘Balkanization of IT,’ driving stakeholders to demand better insight, governance and control of federated technologies to integrate adoption, usage, monitoring, security, cost control and more across shared services.”
1. Enterprise DevOps will grow rapidly
The DevOps revolution has taken IT and app development by storm in recent years, but most of the traction has been in startups or smaller businesses that have more freedom to shift gears and embrace new concepts. Enterprise adoption has been growing, but at a much slower rate. Andi Mann, chief technology advocate for Splunk, said, “In 2016 we will see rapid acceleration of DevOps with 50 percent or more of large businesses finally adopting ‘Enterprise DevOps’ — a pragmatic, scalable approach to adopting the culture, process and technology changes at the heart of DevOps.”
1. The singularity approaches
OK. That may be an ominous over-statement. In a nutshell, Sumo Logic’s Beedgen claims that the line between machines and humans will continue to fade. The foundation of this is automation—leveraging technology to perform mundane or routine tasks and sift through mountains of data exponentially faster than a human can. As artificial intelligence evolves, though, the scope of what can be automated expands as well. “In automating processes, a human can today wield the power of a legion of operators with a single hand,” stated Beedgen. “Likewise, machine learning will increasingly be adopted for its usefulness in augmenting human understanding of complex interaction and large data sets by uncovering the unknown unknowns.”
1. Automation and analytics will improve IT service health
DevOps and microservices have transformed many areas of development and IT infrastructure, but many organizations still rely on decades-old monitoring tools, arcane metrics, and specialized admins dedicated to managing the cumbersome process. Splunk’s Fitz thinks that is getting ready to change. “In 2016, IT analytics will be liberated as easy-to-use dashboards will correlate all sources of service health—legacy tools, APIs, mobile apps, Web logs, wire data, containers and more—while predictive analytics will automatically respond to prevent quality issues.”
1. DevOps is here to stay
From the point of view of DevOps practitioners or companies that have adopted DevOps tools and practices, this may seem both painfully obvious, and yet still be good news as well. Christian Beedgen predicts not just that DevOps is here to stay, but that the importance of DevOps and the sphere of influence affected by DevOps will expand. “Increasingly, DevOps will permeate all manner of businesses that rely on information technology to compete. Bimodal IT (the practice of managing two separate, coherent modes of IT delivery) will become the equivalent to “private cloud”—the last forced resistance of the old guard. Both will be forgotten in a short number of years, as public cloud and lean and agile processes dominate the businesses that still compete.”

There you have it. Now you can enjoy your holiday break in peace and come back in 2016 with an idea of what you have to look forward to in the year ahead.
December 15, 2015
DevOps and IT Support: 4 Principles to Keep Your Team Ahead of the Curve
Here’s the million-dollar IT question: what is DevOps and why does it matter to IT support?

Short answer: DevOps advocates collaboration between Development and IT Operations. It’s about constant communication, breaking down silos and a culture of collaboration. The DevOps movement has made great headway and many companies have adopted and are employing DevOps practices – Netflix, Amazon and Google to name a few.

But where do IT support teams fit into a DevOps environment?

When developers launch new features and services, it’s the support team that deals with issues and interactions with frustrated customers. Change is often the enemy of stability. But because innovation can’t be stopped, IT support teams need to catch up. That means making their operational environments resemble the way their software is developed. The problem is that IT agents and developers have different methodologies, work in different systems, and practically subscribe to different technology religions.

The silos only exacerbate the challenges that the support team ultimately faces:
- Scattered data limits incident and problem understanding.
- Deployment failures have a high risk of downtime. And too often, changes are made that IT support doesn’t know about.
- Support feedback often represents customers, and doesn’t always get communicated to development, product owners and product managers.
DevOps is ultimately a culture change. When all teams commit to a collaborative approach and common goals, it can shift the entire mindset from silos to working together to deliver better services. Agile, Lean and Kanban are popular approaches that facilitate rapid iteration, learning and information sharing. This means you shorten the feedback loop necessary to drive business agility.

CAMS, DevOps and IT support

How do we put DevOps principles into practice for IT support? Over the past 20 years, IT has built up a strong governance model based on ITIL and that can’t be forgotten overnight. While putting new practices isn’t easy, the best way to make big concepts real is a model that covers people, processes and systems. Luckily the CAMS mode coined by Damon Edwards and John Willis at DevOpsDays Mountainview 2010 [1].

It stands for Culture, Automation, Measurement and Sharing.

Principle 1: Culture

The model leads with culture, as it’s the most important facet to DevOps. Support teams learning from and supporting DevOps practices requires a cultural shift. While command and control is comforting, DevOps calls for speed and agility. Daily or weekly rituals can facilitate this cultural shift. Some rituals you can apply to your IT support team include:
- daily standups
- chatrooms
- scrum, sprint planning and retrospectives
- kanban frameworks
Be sure to focus on results, not just the activities. It’s not about the number of changes but the results of those changes.

And finally, understand team behaviors. Culture is a result of habits. You need to understand how your team works together, and what you can improve.

Principle 2: Automation

To move fast, you can’t do everything by hand. Automation is another principle of DevOps, but is most effective when it supports the people and processes in place.

Go from trapped knowledge to shared knowledge
- Known errors should be tracked at the time of deployment and communicated to support. Sound simple? You’ll be surprised how often it doesn’t happen.
- Support should link articles to incidents and problems, so everyone understands what’s causing the issues
Shift from orphaned tickets to linked tickets
- All teams should be responsible for ticket linking and documentation- including the service desk, admins, developers, and operations, so everyone knows what’s going on.
- Link incidents, problems and requests to all associated changes and releases.
Don’t just deploy, but support
- Continuous integration, continuous delivery, and continuous deployment are core to the First Wave of DevOps – thinking about the performance of the overall system. Support needs to be prepared to support, operate and ensure their ability to restore services. They need to know what’s being deployed and when, so they can give the right answers to the customer.
Help with testing
- Automated testing is key to a DevOps environment. Every time a developer commits code, they need to make sure all the code is operating the way it is expected to. If it breaks, tests can catch failures before anything is deployed and delivered. Support teams hold data about various customer environments that should be continuously fed back to testing.
Keep up with monitoring data
- Proactive monitoring helps developers stay ahead of problems before they snowball. In the same vain, support teams can identify customer problems faster by keeping aware of proactive monitoring and linking it back to their own incident management systems.
Principle 3: Measurement

Keep track of metrics that matter. Once you’ve established your individual and team goals, It’s important to measure how you’re tracking to those common goals. How long does it take to recover from failure? Try measuring ‘Mean Time To Repair’ (MTTR) and whether it increases or decreases over time. Here are a few other metrics your support team should keep in mind:
- How many repeated issues do you have? Measure repetitive incidents and how often they occur in a period.
- Are you meeting your targets? Measure service level targets in your service level agreements and operational level agreements in the context of deployments.
- What are your costs? What’s your cost per incident, along with the total cost of support? Take this in context with downtime costs. In some cases, increasing support costs may be the best thing to do to minimize impact to the business.
Principle 4: Sharing

And lastly, when it comes to sharing, ask yourself these questions:
- Does the organization understand what we’re doing? Do we share that understanding?
- Do all teams understand each other’s goals, hardships and priorities?
- Does your team’s outputs of processes flow with other team’s processes?
- Is knowledge shared across teams? If it’s in a system, is it accessible?
- How is your team collaborating with others? Are there unbarred channels for communicating?
- Are the right stakeholders kept in the loop of your projects, even if they’re not in your immediate group?
- Do your IT groups – development, operations, testing, support and even the business – share ownership of successes, failures and learnings?
- Do teams share tools and information? If not, do those tools integrate with each other?
Faster, better IT service is about culture, sharing and tools that enable your teams

A collaborative culture is critical to success in DevOps. Aligning support and development can help your organization understand what services the business wants and needs to be productive.

About the Author/Sid Suri

Sid Suri is the Vice President of Marketing for JIRA Service Desk. He’s worked in various technology marketing roles over the last fifteen years at Salesforce.com, Oracle (CRM), InQuira (acquired by Oracle) and TIBCO Software. He has an MBA from the Haas School of Business and a bachelors in Economics and Italian from Middlebury College. He lives in San Francisco, California.
December 9, 2015
Carving through the hype – take the DevOps maturity assessment

As anyone who spends time with their family over the holidays can attest, the notion of maturity is a highly subjective concept.

Whether it’s sizing up the next generation’s advancement or defaulting to time-honored inter-sibling behaviors, annual get-togethers offer tangible proof of what’s new and those things that likely won’t ever change.

In the domain of DevOps – an umbrella concept that encompasses a massive array of processes and supporting capabilities – gauging the maturity of your organization’s standing is also a highly interpretive task.

Given that “DevOps” means something different to nearly every organization, based on its unique makeup, methodologies, tools, and most importantly, business goals, attempting to measure progress against any outside model may seem an effort of limited value.

That said, as an increasing number of organizations seek to embrace DevOps, and growing recognition that well-worn stories of leading-edge “unicorns” don’t relate closely to most people’s real-world expectations, many teams are left wondering where their own efforts stand, especially in comparison to their peers.

To help acquit this pursuit, and allow organizations to scope their progress, CA recently launched the second iteration of its online DevOps Assessment.

Split into three short surveys covering core limbs of the DevOps tree – “Develop and Test”, “Release and Deploy”, and “Manage and Monitor” – this free tool offers detailed insight into how your organization’s maturity stacks up to that of other practitioners.

For instance, in the domain of development, how well do your teams fare in defining and managing requirements, and then presenting them to others in a structured manner? Related to release cycles, are your teams truly meeting business-driven expectations for more frequent application deployments, shorter lead times, and higher success rates? Or, when it comes to applications monitoring, are you able to distinguish the most critical and recurrent problems and how they truly impact business services?

These are perhaps some of the most fundamental questions that one could apply to measurement of DevOps adoption in general, but further investigation attempts to take a closer look at more finite, underlying issues.

Are your development teams accurately replicating realistic conditions from production systems to create virtual services and testing scenarios? Are your application release deployments fully automated across development, test and production environments? Can your teams monitor complete business services that link user experience, transactions and applications to infrastructure to pinpoint any root cause issues?

The best thing about the survey is that it immediately provides visibility into how your answers compare to those of previous respondents. There’s no fuzzy math or editorial interpretation (though we may highlight trends in a report at some point), merely instant validation, and the option to create a customized report card charting where your efforts stand in relation to those of your peers.

If you’re wondering precisely just how to achieve your long-term DevOps goals comparative to where you stand today, it might even be worth leveraging some of the involved questions for the purpose of evolving process and methodology.

The only remaining questions are: Do you think that you’re ahead of the curve? Are you afraid to find out just how far you’re behind the crowd? And most importantly, what do you have to lose?

Take the survey today and get a real-world fix on your overall level of DevOps maturity. Guaranteed it’s a far more informative activity than debating family and friends over whose turn it is to carve the turkey, or wash the dishes.

December 3, 2015
APM Prepares Prep Sportswear for Promising Growth

Prep Sportswear kicked off in 2003 and appeared either in the Inc. 500 or Inc. 5000 list of fastest-growing private U.S. companies six years in a row, from 2009- 2014, according to online data from Inc. Magazine. The custom sportswear vendor has achieved 130-percent growth over the past three years, says Mike Jones, Director of Technology, Prep Sportswear.

Prep Sportswear clothes more than 200K US schools including most middle schools and high schools with custom-printed sportswear for their school sports teams. Prep Sportswear maintains a catalog of each school’s logos so that teams can create their own shirt or jersey representing that institution and team using its colors and logo or mascot. “So you can create a jersey and for me it would be for the class of 1986, Lake Washington High School, and you would see that Kangaroo on the back or on the front,” says Jones.

The custom-printed sportswear firm serves local teams not affiliated with schools as well as golf courses, colleges, and religious groups. Each customer gets its own store, and Prep Sportswear now maintains more than 2M stores.

Prep Sportswear: Small Team, Big Technology Play

Prep Sportswear produces its own sports goods and maintains manufacturing and attendant technologies while also serving ties to firms such as Active Network and others that use its API.

“We support retail customers, consumers, and our own plants and equipment. On the technology side, we do that with a very small team of ten people. They cover the e-commerce site, our API, and manufacturing needs,” says Jones.

Prep Sportswear programs its codebase and platform using C# and .NET. It also provides for manufacturing components such as ink and vinyl printers and embroidery hardware. Prep Sportswear manages its supply-chain and its related processes and mechanics. “We do it all in-house despite the fact that it is such a large system with a number of components,” says Jones.

How Prep Propped Up DevOps

When Michael Jones arrived at the company, the CTO was deploying releases directly to production from his laptop. “We didn’t have a DevOps team really until Richard came on board,” says Jones, speaking of his colleague Richard Dominguez, Developer in Operations, Prep Sportswear; “before that we were running in start-up mode, just kind of out of a garage until recently.” Richard describes the technical setting when he came on board.

“A lot of the automation they had was not really automation but little scripts that they ran individually,” says Richard Dominguez, Developer in Operations, Prep Sportswear; “when we released something, we had no idea whether it would break.” At the time, there were few tests on software and there were no external tests at all.

The Arrival of Change

With Richard at hand, Prep Sportswear ultimately adopted TeamCity for CI for deploying to its test and production development stages.

The company also added the Selenium browser-based test suite, coded in Java to run the user-based interactions to determine what the user experience is like. “It checks to see that the page and elements load properly when a user clicks a link,” explains Dominguez.

Richard found Dynatrace installed when he arrived but not in use. He revived it as the main APM for front-end monitoring. “We also use another tool a lot for our internal monitoring of infrastructure on the back-end and that tool is PRTG,” says Dominguez.

DevOps Done Right Brings Cheers from the Sidelines

The combined revenue from all channels last year was about $23M, says Jones. “Since Richard came on, we’ve seen a decrease in time to resolution of more than 50-percent just by having the APM and other tools,” says Jones, with reference to Richard’s revival of Dynatrace and addition of the other aforementioned tools.

Up times have improved by .50-percent as well. “That stems from simply being able to find an issue and resolve it faster. For an e-commerce site where we don’t have any sales people, it’s just the website, that additional up time is very important to us,” says Jones.

October 29, 2015
Survey says network agility is still immature

At least that’s the word from SDx Central’s Cloud Management, Orchestration and DevOps 2015 Report in which 76% of respondents said the network was the most immature component in the cloud management stack.

Now to be fair, the network is one of the last components (I’m looking at you, security) to get on board the “automate or die” train, which has already left the station. That’s partly due to pressures from increased DevOps adoption in general and more specifically thanks to greater enterprise penetration which is continuing to up the amperage in the data center with respect to velocity and ease of deployment of all the network bits, bytes, and pipes needed to support each and every application.

It’s also partly due to the complexity of the network environment and the blast radius of potential errors or misconfigurations as well as the broad set of differences in the object models (and therefore APIs) used to configure, manage, and automate provisioning of network services. The more features and capabilities, the more differences between them.

OpenStack manages this difference by forcing standardization; that is, it defines the interface for LBaaS and if you want to be a part of the stack, you have to conform. Cisco ACI manages the difference by taking a declarative policy-based approach that let’s plug-ins define their own interfaces and expose them to ACI in a standardized way. VMware NSX and related technologies take yet another approach, leveraging its ecosystems’ APIs to integrate and orchestrate the network.

All are relatively new to the game (OpenStack may be the exception but even its network implementation has lagged far behind other components of its system) and thus the perception that the network is the most immature in the cloud management stack would seem to be reflective of reality.

But at least it’s heading in the right direction.

The network was going to have to be included at some point in this grand journey to transform the data center through automation and orchestration. At some point Agile and DevOps methodologies being embraced by dev and ops were going to start paying off in the form of more applications with more frequent deploys. Eventually that was going to overwhelm the network (and its staff) and give them cause to start figuring out how they were going to transform the network from the static and very fixed methods of scale and management to a more modern dynamic and fluid method.

Eventually is now. Whether it’s DevOps expanding into the network or SDN or cloud management stacks, the network has to mature in terms of its orchestration and automation capabilities if organizations are going to continue on their journey toward the agile network.

We’re still trying to move out of the first phase in that transformation; toward a culture inclusive of measuring and standardization with templates (or configurable policies, if you prefer). We’ve got a long way to go toward a sustainable, agile network and just as it’s taken years for Agile to truly envelope most projects within the enterprise, it’s going to take years for the network to get from where we are today to where we suddenly want to be yesterday but accept that we won’t get there till tomorrow.

October 21, 2015
Microservices, Hyper-Complexity, and Automation
Imagine building a human being one DNA codon at a time. Sure, you might be able to build a pretty awesome individual. But it would take a long, long time. And you’d only be able to build one. Worse yet, if you didn’t get your person exactly right on your first try, it would take you another long, long time to come up with Version Two.

Unfortunately, this is exactly the position many companies are putting themselves in with microservices.

Yes, microservices theoretically enable companies to adaptively recombine their digital DNA to build the increasingly awesome apps. But without better automation, they won’t be able to do so very quickly—or very reliably.

Hyper-Complex DevOps

Microservices introduce significant additional complexities into the DevOps lifecycle. Each microservice has its own data mappings, APIs, latency/performance dependencies, etc. So, while microservices may offer genuine advantages in terms of adaptability and re-use, they also multiply the total number of discrete DevOps tasks—especially vis-à-vis test/QA—by at least one full order of magnitude.

This is a serious problem for shops that aren’t sufficiently well-automated, because any task—or, as is much more often the case, any handoff between tasks—that is not well-automated will quickly turn into a bottleneck.

The resulting adverse consequences of insufficiently automated microservices DevOps can thus include slow delivery, excessive cost, and mistakes. Worse yet, chronic bottlenecking will likely create a pervasive organizational disincentive that fundamentally undermines the microservices initiative itself.

What Automation Entails

Thorough, effective DevOps automation is, of course, more easily aspired to than achieved. To implement the kind of end-to-end DevOps automation required for microservices success, organizations have to get three things right:
- Process mapping. Effective automation requires an exhaustive definition of process. This exercise is not for the faint of heart—because most organizations have many pockets of ad hoc process at various points in the DevOps cycle. Test/QA tends to be especially variable based on factors such as the delivery pressure teams are under at any given time.
- Automation technology. Development teams often use scripting and other technologies to automate various DevOps tasks. An overly fragmented automation environment, however, can be difficult to orchestrate and manage. So the move to microservices may necessitate a more holistic integrated automation environment.
- Leadership and culture. A well-automated DevOps process alone is insufficient for microservices success, because it is unfortunately not true that “if you build it, they will come.” On the contrary, a strong leadership commitment is essential for changing the way people work.
The application economy is putting development organizations under intense pressure to be more productive and responsive to ever-changing market dynamics. Microservices offer one possible way to respond to that pressure. But microservices won’t work without a much more rigorous approach to DevOps automation. IT leaders may therefore want to focus on automation first and microservices second, instead of the other way around.

Unless, that is, you only want to build one app using microservices—and then maybe build a slightly better one a few months later.
October 19, 2015
Continuous Deployment is king of the DevOps hill

One of the promises of DevOps is continuous deployment. DevOps is, first and foremost, a matter of shifting the traditional business culture more than the tools and technologies that enable it. Once that culture exists, though, one of the defining aspects of DevOps is automation and continuity.

Earlier this year I wrote about the seeming ubiquity of “continuous” solutions throughout DevOps:

If DevOps is the buzzword du jour for tech, then continuous is the buzzword du jour of DevOps. Everything is continuous. Continuous development and continuous testing lead to continuous deployment and continuous delivery, which requires continuous support. Continuous monitoring produces continuous integration and continuous change. Continuous security results in continuous incident response…or vice versa. To top it off, all of the continuous activities continuously feed each other to drive more continuousness in some sort of DevOps Mobius strip of continuity.

Taking a closer look at all of that continuous continuousness, however, continuous deployment stands out as one of the prevailing goals of effective DevOps. Continuous integration, continuous delivery and the other continuous tools and processes are all in some way related to facilitating the flow of continuous deployment or monitoring and managing it after the fact—which ultimately feeds back to the beginning of the loop and back to continuous deployment.

Chris Riley tried to muddle through the confusion of continuous integration, continuous delivery and continuous deployment. In his explanation of continuous deployment, he stressed the impact of this symbiotic loop that surrounds it: “Continuous deployment has additional costs, as it relies on instrumentation, to ensure that new functionality does not result in bugs and also on infrastructure that allows effortlessly backing out new features when a defect has not been caught by automated tests.”

I defer to Riley on most things, but I don’t agree with his summation that everyone should do continuous integration, some should do continuous delivery, but only the rare few should strive for continuous deployment.

I guess it depends on what perspective you’re considering it from. From the standpoint of ease or simplicity and the likelihood that an organization can execute it effectively I can see where Riley is coming from. From the point of view of the goal of embracing DevOps, though, I would reverse his order and say that everyone should do continuous deployment, most should implement continuous delivery to facilitate it, and some (or maybe most as well) should adopt continuous integration to drive continuous delivery.

The primary objective of DevOps, though, is to deploy apps. Everything else plays a supporting role.

That said, there is a difference between embracing DevOps with the goal of developing and deploying apps more efficiently and effectively, and adopting continuous deployment tools that automatically pass all changes that pass automated testing and are verified as deployable by continuous delivery tools into production.

A blog post from Puppet Labs explains, “There are business cases in which IT must wait for a feature to go live, making continuous deployment impractical. While application feature toggles solve many of those cases, they don’t work in every case. The point is to decide whether continuous deployment is right for your company based on business needs — not on IT limitations.”

There are many acronyms and various interpretations out there for DevOps, so my opinion certainly isn’t gospel on the subject. What do you think? Should continuous deployment be the primary goal and centerpiece of DevOps automation? If not, which of the many “continuous” solutions would you suggest as the central focus?

October 15, 2015
Digital Disrupters – Building a DevOps Leadership Profile
While the advancement of DevOps methodology remains fairly nascent for many organizations, an ever-expanding class of leading practitioners are evolving a wide range of related best practices.

Beyond the well-worn concept of “unicorns” – the scant few organizations that stand as mythical examples of DevOps nirvana – this broader set of early adopters have begun to pull back the curtain on their efforts, lending further detail to the strategies that have bred this current model for success.

Last week, industry analysts Freeform Dynamics published valuable research that highlights the ongoing work of such DevOps innovators, dubbed as “Digital Disrupters” for their accomplishments in pursuing larger digital transformation. The report “Exploiting the Software Advantage” – sponsored by CA Technologies, outlines a critical set of initiatives crucial to navigating the DevOps landscape.

What are the specific hallmarks of those organizations leading the charge? Where are others struggling to catch-up? Many of the Freeform Dynamics survey results, based on 1442 respondents, reaffirm widespread DevOps observations, while other findings serve to shed new light, including:
- Everyone builds software: Perception is, in fact, reality and to deeply engrain digital transformation, Digital Disruptors are increasingly identifying with this mantra. While 95% of leaders have committed to becoming more app-centric, only 51% of mainstream organizations have adopted this persona. Similarly, 84% of Disruptors directly identify as “software companies”, regardless of industry, compared to 40% among the masses.
- Transformation is required: This point may seem obvious, but it certainly bears repeating. A whopping 92% of leaders have actively moved to break down traditional silos between Dev and Ops, with 51% of those organizations having already completed this work. By comparison, only 18% of mainstream organizations claim to have done the same, supported by 59% who at least share this more collaborative DevOps vision.
- Commitment is critical: This may also seem like a no-brainer, but the numbers bear out this requirement. Some 77% of Disruptors have agile development methodologies in place, with 70% having “broadly” implemented DevOps activities. For mainstream organizations, these metrics rate at only 31% and 21%, respectively; leaders are going all-in and appreciating the benefits, and those that follow will need to do the same.
- Experimentation is omnipresent: To garner the benefits of continuous delivery, embracing trial and error is a must. Among Disruptors, 93% recognized the need to “experiment and fail quickly on the road to success”, compared to only 58% of mainstream organizations. Some 95% of leaders have already shifted from fixed product cycles to continuous innovation, while only 56% of the mainstream have made this leap.
- APIs are everywhere: Ubiquitous integration, both in-house and in connecting to third parties, has only increased emphasis on the API. Disruptors are resoundingly focused on API management, with 68% using APIs to integrate third party services into their applications, and 66% to offer external access to systems and data. Mainstream peers compare at only 26% and 24%, respectively, denoting a sizable API management gap.
These are only a handful of the most dramatic and obvious areas where DevOps advocates have clearly re-architected their approach, separating themselves from those organizations that have yet to go all-in; it doesn’t stop there.

As highlighted in the research, other areas of differentiation and dichotomy include cross-functional alignment, emphasis on digital channels, use of digital to drive core business efficiency, and even funding of digital investments.

Beyond these key philosophical milestones that litter the pathway to DevOps success, the survey also highlights the vast array of strategic goals that these organizations have sought to accomplish. The results spotlight the massive range of significant business benefits that practitioners believe DevOps can deliver.

Leading use cases include:
- Improved customer satisfaction (49%)
- Better workforce productivity (44%)
- Improved customer experience (41%)
- Improved competitive differentiation (38%)
- Better agility / speed-to-market (37%)
- Opening up of new markets (35%)
- Creation of new business models (30%)
- Creation of new routes to market (26%)
If one were to take those outcomes out of context, unrelated to the impact of DevOps, it’s fair to say that they read like a generic list of leading business initiatives – further underlining the movement’s overall relevance.

That’s not to say this is all accomplished easily. Report respondents detail hard won victories that required significant time and investment to achieve all this DevOps-driven advancement. After all, the notion of “disruption” doesn’t exactly evoke images of ease or comfort.

Click here to read the report for yourself – there’s a lot more to be considered. Namely, how well your organization measures up in affecting digital transformation in its own right, and where it can take steps to move forward.
October 14, 2015