devopsy.kinsta.cloud

Tag: development

Why a Dev/Test Cloud Is Not Optional

How do you create a private Dev/Test cloud to standardize processes, tooling and environments across teams? Recently, I participated in an online panel on the subject of, “Creating an Internal Dev/Test Cloud,” as part of Continuous Discussions (#c9d9), a series of community panels about agile, continuous delivery and DevOps.

Continuous Discussions is a community initiative by Electric Cloud, a provider of continuous delivery to businesses including SpaceX, Cisco Systems, GE and E*TRADE. I am not an employee or vendor of Electric Cloud.

Below are a few insights from my contribution to the panel:

What are your thoughts on Dev/Test cloud?

It’s not a question of why—You don’t really have a choice. I don’t think that this is an optional thing. If you’re moving to this modern software delivery, you’re going to have to build this environment. If by chance you’re already doing some of the continuous integration continuous delivery (CICD) processes without it, it’s not going to last long. If I were to take a slightly different angle, I think the interesting question is where? So where is your Dev/Test cloud?

I think of Dev/Test as a conceptual thing; it’s your factory. If we talk about it from a singleton point of view—as one environment, for example, or one integration environment that it is not going to support microservices and some of the future architectures—that’s when you might add multiple Dev/Test clouds. For example, if you think you’re going to have your browser grid on-prem or intermingling with your other integration environments, that’s a lot of work, and that’s a lot of maintenance. It’s just not a good idea. But at the same time, it’s actually amazing if you’re doing anything with artifact repos. Even though some of these artifacts are really small—and Docker images can be very small, if those repositories are not on the same network as the rest of your Dev/Test—it could be really slow pulling assets down. So that’s what can be brought on-prem. I think the bottom line is that you’re going to have to get there no matter what. It’s just going to happen. I think how you set it up is the interesting question, since there are a lot of options as far as how to do that.

What are the challenges of creating a Dev/Test cloud?

I think the first challenge is parity between your Dev/Test environment and your production environment. Replication would be insane because you’re talking about data centers, a lot of overhead, and it reduces flexibility. And everything we talked about before—the ability of developers to spin out Dev/Test environment and integration environment on a whim. However, there are things that you can absolutely consider, which is consistency between all your configurations, making sure that all your frameworks are consistent because chasing down bugs that cross applications throughout the system are very hard to deal with. But there is always the need to address that.

And we’re even talking about agents sitting on your nodes. I mean, if you’re running some tooling on your servers, it’s good to know what the potential impact of having those, or not having those on your Dev/Test environment could actually be. I’d say that parity is probably one of the biggest challenges. And then the other challenge is just knowing who owns it, because I think that the best scenarios I’ve seen are when you have a steward of the Dev/Test environment. It could be DevOps, it could be somebody else who’s responsible for maintaining the strategy of what you do with those environments versus it being a wild west. So they don’t own it, but they facilitate making sure that parity exists, and facilitate making sure the automation and the provisioning and everything is as efficient as possible. Developers shouldn’t think about that. Dev/Test should be more of an offering. At the bigger organizations, that’s exactly what they do. They’ve created a shared services or cloud services division that is responsible for setting those up.

What are the challenges you see people running into when they’re trying to create a Dev/Test cloud?

It is a huge habit of developers to build locally, and I think that most developers I tell to stop doing that complain. It is a big problem, just from a pure pollution standpoint and the fact that you don’t really know the condition of each developer box, there is no consistency. Hopefully a Dev/Test Cloud is easy enough to utilize that they don’t see the need, and there is some training—I think it goes back to the idea that if you treat it as this singleton thing, you know that some Dev/Test environments live and die within an hour time period. They’re very short. That should be the idea, that you spin it up, put your code and test it quickly, get the IP address. You test it and kill it, and that’s it. You move on and continue coding, and as it moves downstream, you can have a more robust integration environment.

I do think that there are best-of-breed Dev/Test clouds. I think you have the Dev/Test clouds that are great for the spinning up and down a pole instances. And then I think you have past types environments, and I think you have load testing clouds, and you have functional testing clouds. There may be a reason to break them out into different types. The most important thing is that everything is tightly integrated and that there is a strategy. It’s not just off the cuff, because that’s equivalent to shadow IT. Even if everybody knows it’s happening, it’s the same difference.

What Are the Prerequisites for a Dev/Test Cloud?

I think you have to have a steward, somebody who owns it, standardization. You need to have full master scripts, snapshots, images, that represent the de facto Dev/Test environment, and somebody needs to make sure that those are up to date and current so you have consistency across all the locations where these have been spun up. The contradiction is if you are using a performance testing cloud for load testing, a functional testing cloud, where you don’t have control over that infrastructure. In those scenarios, if you’re running Selenium scripts, you’re going outside in, so it doesn’t really matter as much.

How to do it

If you’re starting bottom up, you could think about it all the way through before you develop it. But most organizations aren’t in that position; they’re slip-streaming that into something else, and chances are you’ve already had this concept and you’re already using virtualization, so part of it you probably already have, but maybe not in the most efficient way. Many of the organizations I’ve seen start with something small. For example, they start with a robust integration environment. Druva is an example of a company that’s made this transition. But in all the scenarios I’ve seen, there’s been another catalyst, it’s never just been Dev/Test cloud. There’s always been something else. An architecture change to microservices, changing your product from an on-premises solution to a cloud-based solution, which was the scenario with Druva. There always seems to be something else. I don’t think it has to be that way, but it seems like there are other driving factors that necessitate change and the opportunity to change your pipeline, and they’re driven by the application. I don’t think there’s an easy answer. I think you have to rely on automation, you have to rely on tooling. You can’t be afraid of tooling. Don’t let the tool dictate what you do. You should be telling the tool what to do. If eventually you are a Chef shop and Jenkins, and it dictates what you’re doing, then that’s all you’re going to do. And you’ll never be able to adopt a future tool, so think about that. Inertia is a powerful force.

I have very strong opinions about how Development environments should be set up. Stop building locally, and put it in the cloud.

June 14, 2016
5 Development Trends Reshaping Testing

There’s no shortage of new development trends, but virtual reality, containers, big data, hyperconnected apps and IoT are affecting testing the most today.

If you like change and challenge, it’s a fun time to work in software. Some technology developments, such as cloud computing, have been gaining speed slowly over many years. That steady progression has meant that developers can stay on top of trends easily and be informed about market changes. Other areas, such as containers, have come to fruition much faster without as much notice. Development teams have had to scramble to stay ahead of these trends, hoping to capitalize on a their efficiency gains while staying ahead of the curve. Here’s a look at five important software trends which are having a profound effect on testing.

Containers

Docker made containerization cool and exciting in 2013, even though containers have arguably been around for many years, if not decades. Developers love containers because the technology makes it much easier to set up and install software in different environments, such as in the public cloud or in a private data center. Containers wrap up all of the infrastructure components that otherwise would be installed separately into one easily portable package with a common standard. And since multiple containers can run on the same hardware without creating problems, you can do more with less infrastructure. Typically, containers can move between different environments with relative ease—a nice benefit for multicloud infrastructures. Containers have evolved quickly from an experimental platform to a mainstream infrastructure technology, with many major businesses taking advantage of containers to deliver their product at scale.

Testing considerations: It’s hard to find much at fault for containers when it comes to facilitating software testing. Because of the ease in setting up test environments and moving code from one environment to another during the production cycle, testers can work much faster. Before containers, there was a lot more waiting around by both developers and testers. When it takes two weeks to set up an environment for a testing cycle that took two hours, teams would tend toward less frequent releases or would do less testing. Now, it is a matter of minutes to set up a testing environment. This is great for companies and development teams that value testing and want to test more frequently with greater access to a variety of environments.

Another boon with containers is stability. In the past, bugs sometimes appeared due to configuration errors from setting up dev/test environments. Given the plug-and-play nature of containers, there’s little if any concern about the proper configurations being completed before a testing environment can be used. Security has been an ongoing concern with containers, especially in highly regulated sectors such as health care, government and financial services. In those cases, companies may have to support two versions of the code—one in a containerized platform and one in a more traditional virtual or bare-metal environment. We believe this will be a short-term situation as containerization technology matures and becomes adopted across all industries.

Big Data

Whether you want to use this vague and overhyped term or not, big data is simply a new way to refer to larger than normal data sets. Big data became a big trend as the hardware and software to store and manage it got a lot more powerful and cheaper, and the cloud shift helped to centralize data sets into one location. Some large companies and research organizations are seeing interesting results from big data projects already, whether in more targeted drug development (precision medicine) or just-in-time environmental data for farmers. Now, every Tom, Dick and Harry is trying to get into the big data market, which means there are too many solutions to count. A natural weeding-out process will occur over the next few years, as the prominent big data players and NoSQL standards converge into two or three leading providers.

Testing considerations: Big data applications bring value through the way they manage and/or analyze data, not so much through the user interface. Therefore, testing for big data applications is going to require more technical, database-oriented skills. Those skills are also different from the typical relational database skills of the past, given the fact that big data platforms typically are designed to manage structured and unstructured data. These are open source tools and systems such as NoSQL and Hadoop, along with large vendor systems such as SAP HANA. Testers in this field will need to be constantly adapting learning these new technologies.

Virtual Reality

Investors have funneled $8.8 billion into the sector since 2012, according to SuperData Research. Google, Facebook, Microsoft, Samsung and HTC are all in the game. The market is segmented between VR, augmented reality (AR) and mixed reality (MR) and consists of both traditional (PC/console) and mobile device applications. VR is a fully immersive, fully virtual experience that mimics reality, while AR and MR are different variations of blending real and virtual environments together. VR applications aren’t just for gaming anymore: We now are seeing the technology being used for product design and collaboration, tourism/travel and entertainment, including film.

Testing considerations: VR is an emerging technology still, with many different types of platforms, no clear leaders and no real standards in place. That makes it more difficult for developers and testers to make strategic decisions around technology. As well, there aren’t any great tools for automated testing in these environments yet. Therefore, the need for manual and real-world testing will add time and cost to the project. Yet, investing in doing this right is critical for ensuring optimal user experience. In virtual reality environments, it’s difficult to simulate experiences, such as testing for motion sickness, without actually playing with the real devices. This is especially challenging with the newer mobile platforms, such as Google Cardboard, which incorporate a viewing device that transforms the mobile phone into a VR headset. That experience simply can’t be simulated accurately on a PC, a common testing strategy for other mobile apps. Companies testing mobile VR apps may need to invest in the purchase of dozens or hundreds of different mobile devices to gain an accurate picture of how the application performs for most users.

Hyperconnected Apps

Web-based and consumer apps comprise the bulk of hyperconnected apps, characterized by having deep, often real-time integration with other applications. Instagram and Facebook, Gmail and Google Calendar, Salesforce and their wealth of apps—these are just a few examples of hyperconnectivity. The advent of REST APIs has facilitated far easier and cheaper integration so any company can build a platform of third-party software in ways they couldn’t have imagined five or 10 years ago. Similarly, with the explosion of the app economy, including open source software, it doesn’t make good economic sense to write 100 percent of code in your application any longer. Product value is increasing the result of how developers can patch together different technologies in innovative ways.

Testing considerations: Given the importance of APIs in hyperconnected applications, a primary testing consideration will be ensuring the performance validity of those connection points. It’s imperative for both developers and testers to be aware of API update schedules and to test those updates immediately for any negative impact on the application. Beyond API testing, load and performance testing are also important considerations for hyperconnected applications, which typically involve heavy and/or unpredictable transaction processing. Overall, these types of testing are much more technical in nature than manual UI testing and will require testers to expand their skill set to optimize value.

Internet of Things (IoT)

The interest in (and hype) for the IoT sector rivals the largest IT trends, such as big data. Initial applications have focused on connecting devices, such as home lighting and thermostat systems, which use sensors to stream data to smartphones. Over time, we will see more device-to-device connections such as a thermostat collecting data from a smoke detector to ensure that the house isn’t on fire if the temperature rises dramatically. Products such as Amazon Echo and Google Nest might be at the heart of many new consumer-based IoT applications, but there is also ample potential in sectors such as utilities, health care, industrial equipment and government (a.k.a. smart city initiatives).

Testing considerations: As IoT applications mature, connectivity complexity will grow. IoT applications can involve many components: the hardware of the smart device or product and the software that controls it, along with the user applications that interact with the product. When troubleshooting an issue or vulnerability, a tester must investigate the applications that collect, analyze and display the IoT data, the various network connections across which that data travels, the IoT appliance’s hardware and its software. Hardware testing must incorporate typical scenarios such as what happens to the user experience when a device starts to run low on battery or goes offline. Testers will be most valuable when they have experience in both hardware and software testing; alternatively, an organization can have two teams of testers but they must be in very close contact. IoT apps should also be configured to collect and deliver more alerts and logging data than mobile apps, due to the higher number of components involved. This will make it easier to prevent, troubleshoot and fix problems when they arise.

About the author/Kevin Dunne

Kevin Dunne is VP of Strategy and Business Development at QASymphony.

June 6, 2016
Data Protection for Operational Reporting Platforms
Making real-time business decisions based on data from social, mobile and cloud platforms is a requirement in today’s world. That’s why an ecosystem of next-generation operational reporting platforms such as Spark, Apache Cassandra, Kafka, Docker, Mesos, Marathon and more have come to life. Enterprises are using these technologies to re-architect their data storage and processing frameworks to achieve lower costs, scalability and faster response times. However, as enterprises onboard business-critical applications on this next-generation technology stack, they need to prepare for failures or errors that may disrupt customer-facing applications.

One of the key benefits that operational reporting platforms provide is real-time visibility, which allows businesses to make strategic decisions based on actionable insights. However, the new applications generate a large amount of data and archaic operational reporting platforms (those based on traditional data warehouses, data marts and ETL logic) are unable to catch up and, as a result:
- Business decisions are based on stale data
- Fragmented data increases the risk of errors
- Multiple copies of data increase storage costs
These issues have driven many progressive, data-centric enterprises to re-architect their operational reporting platforms. In this new architecture, Kafka serves as a persistent message bus that ingests data from relational stores and provides failure resiliency and message buffering. Spark or Spark Streaming may be used for transformation, aggregation and other lightweight stream processing. These processing nodes may be deployed in Docker containers. Finally, Apache Cassandra serves as a distributed data storage layer that provides linear scalability and high-availability. Kafka also enables the ability to add multiple data consumers such as Hadoop/HBase through Flafka.

However, as enterprises adopt operational reporting platforms, they want to ensure that bad data feeds and human error do not result in data loss. Apache Cassandra natively replicates data but that does not provide the capability to go back in time to recover data efficiently and at scale.

As enterprises onboard their critical applications to achieve development agility, scalability and lower operating costs, they must also explore data protection solutions that are specifically dedicated for scale-out databases.
May 18, 2016
‘DevOps is a Lifestyle, Not a Title’

“DevOps is a lifestyle, not a title.” This is a quote from Ron Melanson, of Amazon, and I hope he won’t mind me sharing it here. It struck a chord with me, and this blog explains why.

I have spoken about DevOps on a few occasions. Some audiences are not from a development background, so I explain the “why” of DevOps to them to help set context.

Chances are you are past the “why” stage and into the “how” stage. Even so, it is always useful to think about the “why” part, or even to revisit it—it is important to make sure you really are getting what you want from the DevOps journey.

In my DevOps presentations I vary between the transactional aspects of the “why” and the emotive aspects of the “why.” The transactional aspects are what the tooling can help with, and so that matters to a degree. However, since emotions influence us the most, especially from a lifestyle perspective, it is the emotional aspects that I want to focus on here.

The picture below captures, for me, the essence of why people care about DevOps: they want to be happy at work, and to sleep soundly at night. If you are not where you want to be with respect to those things, then you need to change the way you lead your life.

The DevOps lifestyle is a virtuous cycle

I highlight the “change the way you lead your life” aspect to illustrate contrasts among the differing approaches to DevOps that I have witnessed.

Now that DevOps has grabbed the popular imagination, it is quite common to see job advertisements for people to be part of a DevOps team.

One ex-colleague of mine, hired to lead a DevOps team, expressed some concern about whether that notion actually made sense. He saw his team off in a corner focused on tooling and processes, i.e. the transactional, while everyone else in the organization seemed to be doing what they always did.

When his team suggested a change to a process, or tried to explain how a tool could be adopted, they often were at a disadvantage. The people in development and operations, whose working practices would be affected, had not actually bought into the whole idea in practice, and/or simply did not have the bandwidth to think about it.

Consequently, with little or no time to pay attention to what the DevOps team were doing, there was no engagement from the people for whose supposed benefit the DevOps team was laboring. Thus, there was no driver for changes in behavior. They were all missing the emotive “why” part.

The challenge my colleague faced is common. Hiring someone to do DevOps does allow you to focus on getting people with the skills and experience for the transactional and technical aspects of DevOps. That is necessary, but it is far from sufficient.

If the benefits are to accrue to the people working in development and operations—and, indeed, the wider company and business they serve—then it is those people who have to own the idea, top to bottom.

Owning the idea means rethinking what you are doing, and why—in other words, changing your lifestyle. At least at work, anyway.

May 17, 2016
Balance: The Key to Successful DevOps

In the realm of IT buzzwords, DevOps is in a league of its own. The pervasiveness of the word is matched only by the variety of its definitions. Over time, much like the parable of The Blind Men and the Elephant, it has slowly taken shape out of the void, albeit in a backwards fashion. Associated terms such continuous delivery, integrated quality assurance and deployment automation have helped backlight the beast’s silhouette, but there is much more to discover.

Generally, DevOps is defined by how it is used rather than what it is. This is helpful for those already familiar with the amorphous nature of the movement, but can be a confusing introduction for the uninitiated. To achieve a better understanding, it is important to grasp the underlying need driving the shift to a DevOps model.

Currently, the world of IT is barreling toward a future dominated by increasingly specialized and inherently complex automated solutions. Software is taking a front row seat in ways never before possible. (Insert here)-as-a-service offerings and the demand for instantaneous and consistent deployment turnaround is at an all-time high. The days/weeks required to shuffle a solution between development and engineering is a luxury that most companies can no longer afford and is less tolerated by customers. This level of expectation is something that the traditional IT org chart was not designed to cope with. Tech companies everywhere have felt the strain of this increasing momentum, with many at a loss of how to handle it. Demanding that teams work harder, faster, longer will only get so far. And inevitably, increased intensity results in a rise in mistakes and an inferior product.

Balance: Dev’s Yin to Ops’ Yang

Clearly, change was and is needed. The path from development to operations needs to be streamlined at a fundamental level. However, dissolving the barriers between two longstanding departments is not without its perils. As any IT veteran will tell you, development and operations each have their own cultures, quirks and personalities. But the differences in each help create the perfect balance.

In its purest form, development is the trailblazer, forging into uncharted territory and unearthing hidden gems. Ever the innovator, Dev values progress and change above all. This constant forward momentum is Dev’s greatest asset as well as its Achilles’ heel. The habitual urge to tinker, tweak and create can lead to frivolous or wasted effort. The results of this misguided effort can be seen in retrograde patch releases, hotfixes and other “band-aid” measures needed to undo the damage.

On the other end of the spectrum, the operations mantra can be summarized with the idiom, “If it ain’t broke, don’t fix it.” Pragmatic to the core, stability and consistency are the guiding forces behind Ops. Change requests are usually met with a Treebeard-esque, “Don’t be hasty.” Taken to the extreme, this mindset can quickly lead to an environment running outdated, unpatched software and end-of-life hardware.

On the surface, the differences between these two groups appear to be on the verge of irreconcilability. However, in this diversity the beauty of DevOps emerges as the strengths of each team balances out the weaknesses of their counterpart. Development is free to innovate and create knowing that their efforts will be guided by feedback from operations. Like stabilizers on a rocket, operations is able to focus on maintaining a reliable infrastructure with development providing the forward momentum. Working in unison, this duo forms a unit far greater than the sum of its parts. And with it, a company is left with an agile and stable launch pad from which to tackle the latest wave of IT challenges.

Admittedly, the combining of development and operations into a single cohesive unit is just a piece of the entire DevOps puzzle. However, it is an important piece that is frequently overlooked. Understanding the forces and motivations at work within your teams is a pivotal step on the road to a successful DevOps transition.

About the Author / Jared Perry

Jared Perry is a cloud engineer at iland, where he is responsible for maintaining the virtual infrastructure and a number of disaster recovery solutions. He holds a Bachelor of Applied Science in technology management and a Bachelor of Science in IT administration and management from Texas A&M University.

May 9, 2016
Why Your SaaS Ecosystem Isn’t Delivering

Cloud computing and -as-a-service deliverables such as SaaS have the potential to be transformational, but their value depends on whether you see them as tactical, single-purpose tools or as part of a larger and more strategically integrated ecosystem.

The cloud today is embraced almost universally at some level and seen as an essential component of nearly any enterprise, making it easier to enter new markets, transcend physical barriers and gain access to vital services. The most common, basic approach to SaaS may best be described as a JBOS approach, or “Just a Bunch of SaaS.” In this model, SaaS is used throughout the enterprise, but only as single-purpose tools, deployed to meet the needs of a particular siloed requirement. SaaS in that respect is a quick fix, a Band-Aid, lipstick on a pig—and it has not reached its full potential.

There is a point however, where SaaS and DevOps intersect—that point where an integration framework is superimposed over a SaaS catalog, and enterprise users now gain the advantage of each SaaS being part of a greater whole. At this point where SaaS apps and data become integrated strategically, we begin to see collaboration between IT professionals, developers and line-of-business managers; we see a higher level of automation; a more agile solution that takes full advantage of the cloud model; and finally, an overall cultural shift—all components of the DevOps movement.

SaaS’ Potential

The potential of SaaS is well-known and includes lower costs, agile sourcing and deployment, lower CapEx and more functionality to a wider group of users. But, the greater potential has been lost on many corporate users. IT is moving to the cloud, but is it doing so for the right reasons? By focusing on the “quick fix” aspect of SaaS, companies are missing out on the bigger picture. A more successful SaaS strategy defines the future of the business, using SaaS to stay agile and to align with business needs not just the individual SaaS instances but the entire SaaS ecosystem.

The greater potential of SaaS is achieved when it is deployed strategically, with an integration framework—and then the benefits go far beyond lower costs and CapEx. At that point, SaaS becomes transformational.

Unexpected Results

With all the potential, the rapid uptake of SaaS throughout the enterprise has had a few unexpected results. In addition to a marked increase in siloed information and isolated functionality that results from a lack of integration, the “dark data” phenomenon looms on the horizon, especially as the Internet of Things creates millions of new data points that never before existed. IBM Research’s John Kelly, known as the “Father of Watson,” noted recently at the San Francisco Cognitive Colloquium that 80 percent of data remains dark, and by 2020, the number will reach 93 percent.

With the vast majority of this unused data being generated from SaaS apps and IoT deployments, the unexpected result is an enormously untapped potential. Regardless of what the app’s intended purpose may be, who it was meant to benefit and which department deployed it, useful data is being orphaned at the source. In many cases, that data may have practical uses beyond the original intent, but those uses are never seen because it never escapes its own little corner of the enterprise.

In addition, incumbent integration platforms do not lend themselves to the cloud and lack the SaaS adapters necessary for ad hoc integration and line-of-business usage, and the solutions that do exist still lack tools for governance, API management and IoT integration support.

The Role of IT is Shifting

The IT department’s role is shifting dramatically. The need for agility and flexibility, along with a growing dependence on the cloud and as-a-service software, means IT no can longer continue its centralized role, lest the benefits of the cloud be thwarted. In a recent survey conducted by NetEnrich, respondents’ greatest concerns included IT is losing its relevance (18 percent) and business applications are crashing more because IT is not involved (22 percent).

While businesses recognize the shift, still, 39 percent are facing difficulty integrating traditional IT, cloud and DevOps, and there is no DevOps team in more than half of the IT departments surveyed.

The Solution

With more than half of companies lacking a DevOps team to accommodate the integration of traditional IT and the cloud, the most pressing need is to continue re-evaluating IT’s role from one of build-and-deliver to one of facilitator. Use of an as-a-service integration framework that can be guided by and facilitated by IT but used by ad hoc integrators or developers—whose specialization is not integration but have a need to connect applications and systems quickly from time to time—is the obvious next step to resolving the issues of a SaaS ecosystem that isn’t living up to its potential.

Such a framework can be deployed effectively using a bimodal, do-it-yourself integration approach. Those integration platforms that have begun to emerge may still be inadequate to accommodate the pervasive requirements of this major shift, since they still require IT specialists to manage the integration. The use of integration platform as a service (iPaaS) overcomes this limitation by providing a framework that empowers non-specialist IT users. Specifically, IT’s role as facilitator of this self-service platform comes into play by creating and maintaining the integration platform and a library of connectors, which then can be used not only by the integration specialists, but also by ad hoc integrators and “citizen integrators” or power users within the lines of business. As a result, the inefficient priesthood of IT is then replaced with a new balance of power between IT specialists, IT non-specialists and line-of-business users, encouraging more flexibility of integration in response to different priorities.

The iPaaS approach brings a new level of efficiency to the many new projects that are generating storms of data, such as IoT, and it supports a move away from the older IT approach that lacks the speed and agility required in today’s environment. To make the best use of SaaS, it’s time to rethink our approach and, through universal integration platforms, bring it out into the open.

About the Author/Renat Zubairov

Renat Zubairov is CEO and co-founder of elastic.io, a leading innovator in as-a-service integration.

May 2, 2016
DevOps: Metrics Do Matter
One of my favorite quotes about metrics or KPIs is from Jim Fowler, CIO of GE Capital: “Don’t measure me on cost or traditional IT metrics, but on the metrics of the business.”

It’s a perfect tee-up for me to talk about metrics in the context of DevOps.

DevOps is the movement of enabling a business via technology or the ability to look at business processes and issues to mitigate them or achieve a goal.

The Importance of Metrics

DevOps has as one if its five major principles the concept of measuring (CLAMS: culture, lean, automation, measurement and sharing). DevOps helps the business understand how it uses and benefits from its second major resource after people: technology. This is where key performance indicators (KPIs) come in, though hopefully not KPIs such as this:

This specific KPI was created to show the productivity of developers, using lines of code written as the measure. Could you use or explain this? It’s a great example of metrics or KPIs that are based on bygone times of being complicated, disconnected from business goals, or driving the wrong behaviors by placing quantity ahead of quality.

Catch the Ball

DevOps enables everyone to work together toward a common goal. The DevOps lifecycle approach of creating a strategy, figuring out how to do it, introducing those steps, checking the results and, via feedback (people or tools), improving that strategy or product aids in improving the use and proving the benefit of IT.

DevOps can use a trick from lean (Hoshin Kanri), where the strategy from senior management is passed down to each level (“catchball”) such that each team can create its own way of supporting that strategy. Hoshin Kanri allows every person to set their own way of working to meet the goal. How DevOps is that?

By permission Steve Bell: LeanIT @All rights reserved worldwide

KPIs should be designed to help answer these questions:
- Did we improve or decline?
- Are we on course?
- Does the impact effect our organization?
- Do we need help?
KPIs provoke a Yes/No response and force an immediate action/decision. If not, they are simply metrics that probably support a KPI but do not key to help fulfill a strategy or request.

How Do You Create a KPI?

First, look at a business target. Can you see what’s required to help achieve the goal or remove the obstacle to achieving the goal? Then could you determine how you would know this was happening and whether you could you prove it? All answers have to be Yes/No.

I like making a tree, which also acts as a dashboard to quickly show at the various levels the indicators of performance and their status.

This helps make KPIs such as Cost of Delay, Mean Time between Repair, Defects2Incidents and others relevant to the right people at the right time. You could even be brave and flash the status using red and green: Red = we have a problem and need help to overcome or improve and green = all ok. Don’t hide behind amber—see how using just two colors helps guide easier, faster, better and safer.

KPIs should be collaboration and communication tools. Create and use them wisely to keep your organization in business and on course.

About the Author/Daniel Breston

Daniel Breston is the Chief of DevOps Transformation for Ranger4. Daniel has 30+years leading IT Service, Operations, Applications, Data Center and service partners in management of technology enabling business processes and functions. He has also spent 15 years of consulting and coaching leadership teams across the EU and UK in developing their people, processes and use of technology to deliver technology services better, faster, safer. Daniel has qualifications in IT Service Management (ITIL, COBIT,etc.), Lean and DevOPS.
April 28, 2016
DevOps Days London: The Human Factor

Day 2 of DevOpsDays London 2016 kicked off with a presentation from the legendary Gene Kim. The theme that dominated today’s proceedings was the “human factor,” (fitting, as we finished Day 1 with #humanops) nicely started by Gene’s quote from Nathan Shimek: “We need DevOps to make our work humane.”

The second day delved beyond what makes people happy, posing questions regarding what it is to be human, and why automation is not everything when it comes to DevOps. As Gene said, “DevOps transcends the technology stack we are working with.”

DevOps aims to supplant pain with joy and help people find autonomy, mastery, purpose and deep satisfaction in their working lives in technology. Werner Vogel, CTO at Amazon, is often quoted as saying: “You build it, you run it.” But Gene took this further as he explained how a developer had described his most satisfying working experience as coding something and deploying it and monitoring it and fixing it, too. He also touched on the power of shifting left, using Test Driven Development—and its demand that we write the test script in before we even start to code—as an example. That’s building quality in. That’s cross-skilling and cooperation.

It resonated with the audience when Gene said there was one single question that, when answered, tells us everything we want to know about an organization’s DevOps maturity: “To what degree do you fear deployments?”

We came back to this question later in Jeromy Carriere‘s talk about his experiences of scaling at Google. Going the gym for the first time is painful—we have to build muscle memory, he said. We need to practice these processes until they are like breathing.

Gene made a book recommendation: “The High Velocity Edge,” by Steven J Spear. When we do DevOps we are aiming to transform into a dynamic learning organization. We should not expect but be required to talk about problems. No fear, no blame. Everyone in an organization should be trained to run a blameless post-mortem—or a healthy retrospective, if you prefer. As Claire Agutter put it: “Improving daily work is more important than doing daily work.”

Gene then outlined the DevOps Enterprise Summit London event coming up at the end of June and the extraordinary stories of experiences people share about their DevOps journeys during DOES events. He said: “Often the transformation leader was putting themselves in personal jeopardy.”

I can testify to that. It takes courage, optimism, conviction and tremendous amounts of energy to be a change agent.

The human theme continued into the panel discussion as the COO of Technology at Barclays explained how its command and control structure, waterfall-ruled development and volume of changes resulted in total gridlock. He said that the big challenge is trying to re-engineer the organization to behave differently: “Changing HOW we do our work is the hardest thing.”

Changing human behavior is very challenging indeed; as Joanne said, measurements have to change and we have to focus on customer value. Gareth Rushgrove also majored on the human angle in his talk on DevOps and microservices. He said: “Microservices are as much about people as technology,” and, “Which comes first: the software or the team?”

Essentially, DevOps and microservices help us to make lots of small bets—spreadbetting, if you like—spreading risk and accelerating feedback. And we need small teams to be able to do microservices and move away from monolithic applications.

One particular Ignite talk stood out from the rest, and won the hearts and minds of the audience: @coldclimate from Server Density. Through a personal story of a back injury brought on by prioritizing work over looking after himself, the #humanops message: “You are not immutable, you are not highly available!” struck home hard.

So there we have it. My key takeaway from DevOpsDays London 2016? DevOps is less about technology and automation and more about happy, healthy humans.

April 28, 2016
Does the Future of DevOps Lie in NoOps?

Why DevOps is more a bi-directional rather than a simple top-down approach

DevOps is one of those buzzwords that nearly everyone wants to discuss these days. However, it often is interpreted as an extension of Development’s influence to the detriment of Operations teams.

So much has changed in the past few years that it is conceivable we will see another major shakeup: Could Operations teams disappear altogether? Or to put it another way, is “NoOps” the future of IT? While DevOps promotes faster delivery of new digital services and a blurring of the lines between Dev and Ops, in reality, is Dev consuming Ops entirely?

Before DevOps was called DevOps, the two disciplines were growing further apart. Agility has been driven by Developers since the turn of the century, with the famous Manifesto for Agile Development. Meanwhile, Operations formalized repositories of good practices into frameworks such as ITIL or COBIT, which were implemented by most large companies in the mid-90s. Those two approaches seem to oppose, as DevOps advocates the acceleration and multiplication of changes and ITIL tends to monitor and control them.

NoOps, or the End of Operations?

If you believe everything you read, then Operations is a dying art. But in practice, Development does not want to be involved with infrastructure, while few organizations would trust developers to manage their data centers, whether physical, virtual or cloud. So breathe easy: DevOps does not mean the end of Operations teams.

On the contrary, Operations remains essential throughout the DevOps approach. Production infrastructure is a complex and delicate stack of technologies, often with a discreet mix of modern architecture and historical applications. It often is impossible for Development to replicate these types of environments. And that is why an application successfully tested in a qualification environment often does not work properly in production. However, it is crucial to validate new developments in a representative replica of the production environment.

So is DevOps doomed to constant failure because of lack of continuity between qualification platforms and production systems? Well, this is probably the case if we merely consider DevOps as a simple top-down approach from Development to Operations. Focusing on producing changes at a higher frequency only shifts and grows a bottleneck that sits in production, without producing the added value expected by the company. As a consequence, DevOps must also be seen as a bottom-up process. In other words, if we consider DevOps, we must also take very serious look at “OpsDev” (OK, it doesn’t flow off the tongue, but you see my point!).

DevOps or OpsDev?

For a successful DevOps approach in practice, Development must position itself as a consumer of turnkey infrastructure environments. Operations then adopts an OpsDev approach and provides infrastructure on demand for all steps of continuous integration, from compilation to qualification, through unit testing.

If DevOps is a radical change in how Development works, OpsDev would also revolutionizing common Operations practices. It comes with more agile “declarative” infrastructures (described and built from source code), an even more sophisticated level of automation and the ability to provide self-service infrastructure for developers.

The adoption of DevOps does not mean that Operations will disappear or will be replaced by Development, but rather the emergence of bi-directional agility that needs to be supported by more sophisticated automation. In fact, DevOps implies OpsDev rather than NoOps. Finally, whatever you consider your release process under the ITIL or DevOps angle, isn’t the end goal exactly the same: delivering zero-default service to customers?

April 28, 2016
State of DevOps Survey – The Ultimate Feedback Loop

If there’s any consistent criticism of the larger DevOps movement, it’s that the talking heads and industry gadflies are driving the conversation, not the people in the trenches.

All the thought leadership panels and “how-to” content (including the written and AV materials on this very site) make for great watercooler discussion, but lack the input of those dev, ops and IT management pros who are actually living the DevOps experience, some critics have said.

Well, fair enough. Talking heads are paid to talk. Thought leaders are chosen to lead the debate at conferences and on webcasts. Bloggers blog. Reporters report. This is simply the way of the world. Certain voices are heard via the channels they frequent. It is, as so many people are fond of saying these days, what it is.

So what about everyone else? What of the people reading this stunning prose and listening to these talks while rolling their eyes based on the connectivity of it all to their real-world experience? How can their voices be heard?

Independent of jumping into the blogsphere or applying to speak at said industry events – which you should, btw – there’s one significant opportunity available to anyone who cares to take the time – participating in industry research.

Over the next few business days, citizens of the DevOps world will have their last chance to contribute to the industry’s most widely read and respected research project, the State of DevOps Report.

If you haven’t done so already, take the time to fill out this year’s survey, and make sure that your voice is heard when the subsequent report is published in June. Sure, it takes a little time, about 15-20 minutes; but that’s because it goes to the trouble of exploring the real-world issues that weigh most heavily on today’s practitioners.

Why is this particular project so important, or even credible? Consider that the minds behind the survey and analysis resulting in the 2016 State of DevOps Report are those authors and talking heads that actually get the most respect.

Backed by DevOps Research and Assessments (DORA), the independent think tank operated by a partnership of Jez Humble, Gene Kim, and Nicole Forsgren – perhaps the most respected voices on the landscape, along with Nigel Kersten, the project is categorically aimed at surfacing the truth about DevOps adoption today – not merely promoting the interests of vendor sponsors.

Like previous iterations of the report, the 2016 edition will focus on topics including IT performance, technical practices, culture, lean management, and organizational performance. This year’s survey also examines several new areas, including the ROI of IT performance, security and compliance, and containers.

To further entice participation, project leaders are also offering free tickets to the DevOps Enterprise Summit, along with forthcoming books from some of the industry’s leading advocates to a lucky few who take the time to get involved.

DevOps success is fundamentally based on the notion of increased collaboration and continuous feedback among the people scattered across the SDLC. Likewise, the legitimacy of this research project is utterly dependent on the input of as many real-world practitioners as possible.

If you’re one of those individuals, and you really care whether or not everyone else involved in the advancement of DevOps understands what this all means to you, do yourself a favor and participate.

Much like an application being built using DevOps practices, this quality of this project will be directly determined by the degree to which anyone with insight is willing to get involved and build toward optimal results.

Take the 2016 State of DevOps Survey… today!

April 15, 2016