devopsy.kinsta.cloud

Tag: devops culture

DevOps Gets More Exciting in 2018

As 2017 winds down and the IT industry prepares for 2018, there are a number of DevOps trends that everyone should be keeping their eyes on. It’s not so much that something incredible and explosive is expected in the new year. Rather, 2018 will be a time when DevOps starts to mature alongside other trends important to developers.

These trends don’t guarantee organizations will be immediately successful with DevOps. Most still will struggle with the management aspects of DevOps, especially the need to keep feuding silos within IT working together as one happy group. Assuming developers, QA, and Operations can all learn to get along and act as a team, the technology supporting DevOps will continue to improve and the ecosystem will grow.

Here are some of the DevOps trends that Amalgam Insights is following for 2018:

Everyone Loves Containers and the Ecosystem Grows

Containers have become a common technology in many IT departments. Although the majority of applications remain housed on single servers and virtual machines, container deployments are growing. Containers are an important enabling technology for DevOps—by creating small, transportable bits of applications, containers make it easy for IT silos to work from the same page.

In 2018, growth in container deployments is expected to accelerate as large companies outside the software industry, such as those in financial services and health care, adopt microservices infrastructure. Containers that are especially suited to microservices will continue to expand rapidly.

Increased container growth is also due to an expanding ecosystem. More companies have entered the market with products necessary to large-scale production applications. Products now exist for securing, managing, deploying and enhancing container services that weren’t available a few years ago. What’s more, containers are now available in all the major cloud services including Oracle Cloud, Microsoft Azure, AWS, IBM Cloud and Google Cloud. Containers are becoming enterprise-ready and large enterprises are feeling more confident using them.

Takeaway: Growth in the ecosystem contributes to growth in container deployments while expansion of containers entices companies to enter the ecosystem. Containers have reached mainstream normalization and we expect to see fast expansion throughout 2018.

Kubernetes Eats the Container Orchestration World

In the container orchestration space, Kubernetes has become the tool of choice. Orchestration is the deployment and management of container clusters and is necessary to deploy them at scale. Kubernetes, originally a Google open source project and now managed by the Cloud Native Computing Foundation, has become the most popular orchestration tool available. That became obvious when Docker decided to distribute and support Kubernetes, despite having invested in its own orchestration system called Swarm. CoreOS, Microsoft Azure, Red Hat OpenShift, Docker and others have their own enterprise distributions as well.

Kubernetes brings to the container ecosystem a relatively easy way to manage containers at scale. In large container deployments, containers are constantly being spun up, shut down, moved and copied. Once container clusters are in place, they must be monitored for security and service failure.

During development, there will be a number of clusters that must be managed for development, test and final release to production. Doing this manually is close to impossible when even a moderate number of containers are in the DevOps pipeline. Kubernetes provides the tools and workflow to simplify these tasks.

Takeaway: As container production clusters grow and as they are integrated into the DevOps pipeline, the need for orchestration becomes acute. Kubernetes has won the game and is now the tool of choice for orchestration.

The Pipeline is Getting More Integrated

DevOps is often described of as a “pipeline,” In this context, a pipeline is a linear workflow that moves from build to test to release. As DevOps becomes more common in organizations, the pipeline grows to include additional test phases, monitoring and continuous improvement to create more of a cycle than a linear flow.

One of the great impediments to making DevOps a reality is a lack of tools that support the entire pipeline. There are plenty of tools that help build applications, such as IDEs and code repositories from companies such as IBM, Microsoft, GitHub and Oracle, as well as dozens of open source products. Testing is well-supported with test data management and test plan management tools from companies such Atlassian, Informatica and Oracle. Release management and deployment products are widely available from Atlassian, Microsoft, IBM and many other companies. The problem lies not with the support for the individual steps in the pipeline, but with the integration of these tools to provide a complete end-to-end workflow. This creates a disjointed workflow, prone to errors and supportive of silos instead of collaboration.

This is changing, however, and announcements late in 2017 herald toolsets that can provide a continuous DevOps pipeline. A great example of this is the newest versions of Microsoft’s developer tools that were debuted at Microsoft Connect(); in November 2017. Tying together the Visual Studio debugger with Kubernetes allows developers to test and debug as close to the production environment as possible and then deploy cluster to the test environment. That, plus the upgrades to Team Server, shows the commitment Microsoft has to creating integrated DevOps systems.

Takeaway: While DevOps tools are still more or less disjointed, we are seeing a movement to fully integrated toolsets that enable the end-to-end DevOps pipeline.

2018 is the Year of DevOps—Unless IT Management Gets It Wrong

This coming year could be the year that DevOps truly takes off and becomes mainstream. Those who believe it is already mainstream now are fooling themselves. Management may believe their organization is implementing DevOps, but too many barriers still exist—especially management itself. IT departments remain heavily siloed and independent of each other, as do development tools. DevOps at scale requires, above all else, a collaborative and cooperative approach. Sadly, that does not yet exist in many IT departments. As tools reach maturity and best practices become common, managers become the inhibitors of DevOps. New management structures, incentives and thought processes need to be in place or the money spent on tools is wasted.

Takeaway: All the DevOps pieces are in place except for the breaking down of functional silos that exist in IT departments. 2018 may be the year of DevOps if IT managers can create a new environment of cooperation.

About the Author / Tom Petrocelli

Tom Petrocelli is a contributing analyst with Amalgam Insights. His area of interest is collaboration and new ways of work, developer tools, IT project efficiency, governance and methodologies and DevOps. Most recently, Tom worked for a large, global banking corporation. Previously, he was the research director for Enterprise Social, Mobile and Cloud Applications at Neuralytix. He is an experienced marketing, technology and business executive with 30+ years in the computer technology industry. Tom’s background spans software engineering, systems architecture, IT, product management and marketing and general management. Connect with him on LinkedIn and follow him on Twitter.

December 8, 2017
Cuba Embraces DevOps, Despite Infrastructure Challenges

A few weeks ago I had the pleasure of speaking at the second DevOpsDays event in Cuba alongside some great technologists from across the United States, Europe and Latin America. The conference was held in a beautiful hotel in Varadero overlooking a marina right by the beach. Varadero is approximately two hours by car from Havana.

The first thing I noticed is that Cuba is full of anachronisms. This became evident as soon as I walked out of Havana Airport and realized that the whole “they have some old cars” thing was real. It was fascinating to see that every single car was pre-1950s, some perfectly maintained with great paint jobs and others in worse condition yet still capable of doing their core job as a mode of transport.

Talking technology in Cuba was eye-opening, to say the least. Do not expect to have 3G or 4G data on your cellphone. Do not expect to have WiFi in most hotels (you can purchase prepaid internet credit, but the speed will be slower than what you are used to) and definitely do not expect to have WiFi in a private home. These are luxuries.

But here’s the surprising part: There are companies in Cuba with fully automated CI/CD pipelines achieving 10 to 12 deployments per day and solving real problems, such as the team working under Anesto del Toro at Datys. Cuba even has its very own Docker Captain, who gave a superb demonstration on the value of Docker in front of the DevOpsDays audience.

Now let me remind you, these guys are not working with stable internet connections and they manage to get the job done on a daily basis.

Why am I Rambling on About Cuba?

Engineering is about solving problems. No matter how big or small, it’s about looking at a problem and finding the solution. Now, this is part of Cuban life. If something is broken, they don’t throw it in the trash and buy a new one; they fix it. Their 80-year-old car breaks down two hours outside of town? They get out of the car to fix it and—take my word for this—their friends, family and neighbors will come outside to fix the problem, too. This creates a mindset of solving the problem at hand without fail. This is easily demonstrated when you talk to Cubans about software development: They won’t say they are a Python engineer, a Java engineer or any other choice of language—they are software engineers who understand how development works as a whole. They will use the language required to get the job done to the highest standard.

Like everywhere else in the world, there were huge differences in levels of DevOps adoption between the various organizations in the country. In contrast to the companies mentioned earlier, there also are organizations that have only just heard of these methodologies or are now considering setting up their first CI pipelines. Like many of us, they are still figuring out how to solve their technical debt and modernize their IT Infrastructure, but the key point is that the stakeholders, C-level executives, junior engineers and everyone else in between want to learn. They are open-minded to the principles of DevOps. Every person in the room at DevOpsDays Cuba was focused and listening to what we had to say. This excited me as a speaker and everyone else in attendance. We could see people who really did want to change and transform the cultures within their technology teams and overall businesses to start adopting these modern and proven practices.

The journey ahead is a long one. The infrastructure challenges remain in place and there is much to be learned, but it was great to see such positive energy and eagerness to adapt. I want to thank everyone who attended DevOpsDays Cuba for making it such an amazing experience. An even bigger thanks to the organizers Enrique Carbonell, Rudy Gevaert, Yadier Perdemo, Manuel Oliver, Mike Rosado, Manuel Morejon and anyone I’ve missed. The event couldn’t have been better and it was a genuine pleasure to visit Cuba, I hope to return soon and reconnect with as many of you as possible!

About the Author / Conor Delanbanque

Conor Delanbanque has been recruiting and leading recruitment teams in the DevOps space for the past few years. As well as being exclusively responsible for growing some of the most innovative DevOps & SRE teams in the US and Europe as an external consultant and recruiting partner, Conor also founded the Future of DevOps Thought Leaders Debate, sponsors DockerNYC Meetup on a regular basis and co-hosts ServerlessNYC. You can tweet him at @ConorDevOps, find him on LinkedIn or at events, expos and writing articles on devopsy.kinsta.cloud.

November 21, 2017
Barriers to DevOps Culture Change

In the quest for DevOps, there are two paths organizations can take. One is to start with automation and attempt to get as much as possible out of automation tools. The other is to start with culture change and use tools to address what increased communications uncovers. Both scenarios are happening in the wild, and both have strengths and weaknesses, but there are themes across both approaches that are worth calling out.

DevOps: Culture vs. Tools

An extension of the above distinction is that some IT shops try to implement all changes with tools. After all, a large part of DevOps is the benefits that tools offer, so why not try to reap those benefits without changing the entire culture? This approach meets with limited success. Of course, things improve, since more automation means less room for human error. But there is a limit to how much improvement can be gained without the culture change that allows Ops to ask, “You are slating this project to use BobsEasyDatabase, but we only use the big three. Any chance we can retarget and save issues down the road?” Early and frequent communications can smooth over issues like this and allow a team to do a full-life cycle risk assessment of contentious choices.

Do not let an aversion to culture change limit your DevOps initiatives. It can be difficult to change teams around such that management responsibility changes or is even blurred. But by increasing the authority of a cross-functional team to make the decisions needed to get reliable software out quickly, these issues can be mitigated. It will be an adjustment for many, including middle management, but it is a change that allows the full force of DevOps to improve software delivery.

Silos vs. Culture

In traditional IT, Development happened largely in a vacuum and Operations received the results. The place of security and infrastructure teams was generally with Operations, though they dealt a bit more with development than, say, the systems management team did. The problem was that Development often did what was best for development, with little visibility into how badly it impacted the other teams in the life cycle. The same was true of Operations, which would sometimes “just say no” to any request outside of normal operations without consideration of overall life cycle. And also for Security, which theoretically has veto power for strange implementations in more organizations. These attitudes have to change if DevOps’ strength is to be realized.

Do not leave silos in place that reduce communications between teams. If possible, rebuild teams along application or application portfolio lines so that Dev, Ops, Security and Infrastructure are together discussing what is best for the application and the company. But if that is not possible, regular stand-up meetings to discuss contentious issues such as the security of a given library or inclusion of “Yet Another Database Management App” in a project. Barriers segment work in ways that make the lives of some easier at the expense of the overall project. Break them down as much as your organization will allow.

Driving Home the Changes

In the long run, the point of DevOps is to improve the product and the delivery reliability of the product. Making changes that forward these goals is paramount to success.

Do not let inertia stop culture change. The DevOps Institute has a Certified DevOps Leader program that points out (among other things) how middle management can be the stumbling block in many DevOps implementations. We won’t give away too much of its training, but will say that the organization has good tools for dealing with the dilemma of telling middle management they must meet deadlines while the organizational structure is shifting under them.

There always will be people who believe their role is so unique it does not belong as part of the DevOps process. To make DevOps successful at reducing time to market while simultaneously improving quality and agility, every part of the process must be visited. Quite often, the person whose job is so unique is the bottleneck in the process. If the decision is made to change culture, include everyone, even when it’s uncomfortable.

— Don Macvittie

October 13, 2017
Understanding is Key to Achieving DevOps Culture

Historically speaking, IT has been composed of silos. This has worked out rather well over the last half century or so, as one team developed systems, another kept them running, another handled security and yet another managed the infrastructure. But it was never nirvana. Working well is not the same as cannot be improved.

The Growth of DevOps

Into this static hierarchy of work-performed silos came agile development. Agile addressed shortcomings in traditional development methodologies by breaking problems into smaller pieces and implementing those small pieces quickly. It allowed for a faster check on quality/completeness of those smaller pieces, and tracking the smaller pieces allowed for better prediction of timelines.

Soon, automated testing was thrown into the mix with agile, so developers could receive faster feedback on their changes and address any issues turned up by automated testing while the chunk of code was fresh in their minds.

This lead to teams building more and more of the build/test/deploy process into a system that became known as DevOps. As the change from Agile to DevOps occurred, it became more and more important for communications to happen between teams —Development, Ops and Test—not only to get automation into place, but also to keep things flowing smoothly.

Which brings us to today. Today we have tools to automate just about the entire application life cycle. And while you were reading that sentence, a new tool was released.

DevOps Culture Definition

The problem with this plethora of tools is not that they’re useless—some of them are downright astounding in how they help us develop and deploy software faster, and some increase the quality of the code we are shipping while not increasing timelines. The problem is that in the old, siloed world, these tools are never enough—problems that occur often have Douglas Adams’s famous SEP (Somebody Else’s Problem) field wrapped around them.

This is where DevOps culture comes in. Martin Fowler has a good discussion of culture on his blog, it’s worth a read if you haven’t seen it before. Whether an organization sets out to change its culture or sets out to automate with DevOps tools, a certain amount of culture change is going to happen. That is because lines get blurred relatively quickly if the focus is on turning out high-quality software at velocity.

DevOps culture is teams working together to achieve the goal of delivery in ways that they would not have a decade ago. Communication is king, and is usually the start of culture change. Operations providing immediate feedback when development makes a dependency assumption might be a starting point, then Development starts including Operations in decisions to build in dependencies.

Eventually, whether by design or need, teams become reorganized based on the application or application portfolio they are supporting, and the lines between Development and Operations blur into DevOps, where everyone on a team is working closely to achieve the goal of rapid development and deployment of software.

Culture change can take the form of operational changes, wherein team members start to work more closely together in the absence of major restructuring with the goal of making software manufacturing and deployment better. Culture change also can be formalized, with reorgs of IT to put together teams that will manage an application for the entirety of its life cycle, and Dev/Ops/Security/Infrastructure members are all working together, not as worried about boundaries as in a traditional IT organization. Both are DevOps culture change, though most who specialize in DevOps think the first form is just the early stages, and the second is inevitable in a DevOps shop.

The long goal of DevOps is to increase the rate of new feature development, reduce the rate of bugs and improve response time to serious issues. Culture change is the vehicle that drives these changes. Automation tools can do some, but increased collaboration means increased response rates to all of the above issues, and that’s where true improvements start to show. More advanced DevOps implementations include business unit owners in the feedback loop, giving them instant access to information about the state of the applications they are responsible for and allowing them to adjust as market conditions change. This is another increase in the mostly communications front that provides customers with more reactive/adaptive solutions.

— Don Macvittie

October 3, 2017
Creatures of Habit, Snowflakes and Integration Tax

Donovan Brown of Microsoft, in conversation with Matt Hilbert of Redgate Software, discusses database DevOps’ biggest challenges

When DevOps was first talked about in Flickr’s seminal “10 deploys per day” Velocity presentation in 2009, it was regarded by some as strange and alien to corporate culture. It was the antithesis to the accepted way of doing things, a threat to established and proven processes—it was downright dangerous.

Times have changed. Unicorns such as Uber, Airbnb, Spotify and GitHub couldn’t live without it. Microsoft has embraced it and actively promotes it. It’s become the key to gaining competitive advantage in a technology marketplace that is moving faster and changing quicker than it ever has before.

But what about the database? Why are practices such as version control, continuous integration and automated deployment being introduced to application development but left on the shelf when it comes to the database? The 2017 “State of Database DevOps survey,” for example, found that continuous integration was in place for 39 percent of applications, but fell to 20 percent for databases. What’s going on?

In search of some answers, we spoke to Donovan Brown, Principal DevOps Manager at Microsoft and DevOps advocate. Also known as The Man in the Black Shirt, his unofficial tagline is #RubDevOpsOnIt. He lives DevOps. In fact, you can’t stop him talking about it and enthusing about its advantages.

He describes DevOps as “the union of people, process, and products to enable continuous delivery of value to our end users.” The most important of those three, according to Donovan, is people. It’s also, conversely, the most difficult to fix.

‘People are Creatures of Habit.’

“If you’ve been doing something the same way for 30 years,” he said, “why would you fix it? We all resist change, even if change is for the better, and DevOps requires a lot of change.”

A large part of that needs to come from the Ops side of the equation because, while processes including version control and continuous integration have become second nature to developers, Ops teams traditionally don’t do them on a daily basis. In fact, Ops are often seen as the opponents of change.

Donovan does, however, hold his hand up as being partly responsible for the division. “Historically,” he said, “they’ve had to become the gatekeeper because developers like me 20 years ago were wreaking havoc on production servers, running on there with admin credentials. Their job in my opinion has never been to deploy software for us. Their job is to protect and manage the infrastructure, to make sure it runs as efficiently as it can. That history, though, stays with us because there is so much regulation and red tape to get code from the fingers of developers to the hands of users.”

He sees DevOps as the solution to the problem. “What I’m hoping is that DevOps will help automate what we’re forcing them to do, which was never their job, and rebuild the trust between Dev and Ops. We really need to help Ops people realize that we’re not trying to replace them, we’re trying to empower and enable them to allow us to continuously deliver value.”

It’s not down to Ops teams on their own, however. C-level executives also have a part to play because they often favor a traditional approach to software development, with targets and long term strategies standing in the way of DevOps.

“What I’ve noticed in a lot of companies that are struggling to adopt DevOps is that they’re not getting the backing from the executive level,” Donovan saids. “You have this waterfall mentality from the top that still wants GANT charts and milestones, and things that just do not line up with agile thinking at all. I’ve had the best success implementing DevOps in organizations where the C-level executive got it. When you have a grassroots team who get it, but no one else does, they usually get snuffed out by tradition.”

The database is not separate from Donovan’s goal of people, process and products coming together. Prior to joining Microsoft, he was a Process Consultant and Scrum Master, helping a range of companies introduce DevOps to their development practices. He made sure the database was part of that.

Then, as now, including the database was a challenge, but Donovan worked around it, using the technologies available to allow customers to treat the database just like every other element in their DevOps solution.

He does, however, admit it was hard work. “It’s actually the most difficult part of the pipeline to automate because of the risk of failure,” he said. “You can easily roll back a front end; you can’t do the same for your database because any transactions that have occurred during the deployment could potentially be lost. So, it’s a crucial, integral part of your DevOps pipeline and I challenge any company that claims to be doing DevOps that only does it for the front-end but their back-end is still done manually. To me, they’re faking it.”

‘Infrastructure as code should not be a special snowflake. It’s just part of your solution.’

The key is to remember that the code written to make changes to a database is just that: code. Traditionally, it has been managed outside of and separate to application code, with database changes often managed on-the-fly, at the last minute. Consequently, deployments have become worrying at best, with teams on hand for the expected firefight against downtime.

Include it in version control, however—and continuous integration and automated deployments—and the database can be developed in parallel with applications. Problems are then picked up earlier, deployments become predictable and reliable, downtime and rollbacks are a thing of the past.

What has made this even more appealing is the tooling now available—software that in Donovan’s days as a consultant was only just emerging. As he admitted: “The tooling had to mature to allow us to get over a lot of the hurdles that are unique to database deployments so that we can have everyone incorporated into the DevOps pipeline, not just the unicorns. Everyone should have the power available to them to be able to take their databases and incorporate them as part of their DevOps pipeline. Until you do, you still have that being the bottleneck.”

There is a price to pay, however, if by introducing tools to bring DevOps to the database, companies get bogged down by what Donovan calls “the vendor explosion.”

‘Every vendor you add to your pipeline also adds integration tax.’

Integration tax is the cost to companies, in terms of training and introducing unfamiliar ways of working, to use a new piece of software that resolves a piece of the DevOps puzzle. If that cost is too high by, for example, moving people out of the development environment they’re comfortable with, it betrays the ideals of DevOps. One onerous way of working is simply replaced by another.

Donovan is keen to remove that tax by encouraging companies to work with as few vendors as possible—and preferably those vendors who have already paid the integration tax in advance. By, for example, introducing tools that work inside or alongside existing software, thereby reducing the time it takes to become familiar with them, and making the most of the current infrastructure rather than adding to it.

Otherwise, Donovan concluded, “You’re paying tax for that integration. When we at Microsoft have partners who create extensions for our products, it’s the vendor and Microsoft who pay the tax in advance.”

Finally, I asked Donovan what the biggest challenge was for companies who adopt DevOps for the database. His answer was the most telling.

“If you’re doing DevOps differently for your database and the rest of your system, to me there’s something wrong there. When I think of DevOps, the database is part of that. I don’t need a separate conversation about it.”

About the Author/Matt Hilbert

Matt Hilbert is a technology writer at Redgate Software with 20 years’ experience working for lots of the world’s biggest tech companies – and many of the smallest. He has a particular fascination for emerging technologies and is on a continuing mission to decompile, untangle, and explain techspeak to a wider audience, and excite people about the endless possibilities technology offers.

August 18, 2017
Go Big or Go Home: Total DevOps Guarantees Results
As revealed by a recent survey, only enterprises that implement an “all-in” approach to DevOps achieve the results of best-in-class enterprises seen in the “2017 State of DevOps Report.” Total DevOps approaches DevOps transformations holistically, integrating the full breadth of your DevOps value chain as well as the many stakeholders across your organization that must be engaged to ensure broad, enduring success.

Trace3’s Total DevOps transformation approach uses the following steps:
1. 1. Identify the DevOps Leadership & Core Team. Your Leadership and Core Team members should be open to change, early adopters and innovator-types.
  2. Together as a team, set your true north and determine your big picture business goals for your DevOps journey. End-to-end pipeline lead time, release throughput, release quality, security, satisfaction and productivity are some examples of DevOps goal areas.
  3. Map the end-to-end (E2E) value streams specific to your chosen DevOps goals. Focusing on the goals your organization has for DevOps will ensure your team and organization are aligned and focused on outcomes.
  4. Identify bottlenecks for each value stream that will most affect each goal. During this step, gaps between current state and desired state are made clear.
  5. Dig deeper into the processes and further explore bottlenecks and gaps. Concentrate on deeply understanding how the relevant bottlenecks and gaps affect lead time to define specific improvement opportunities.
  6. Define solution strategies and tactics that align with goals and address improvement opportunities identified during analysis of current state. Total DevOps takes a comprehensive perspective of people, process and technology to define strategies and tactics.
  7. Develop an overall DevOps road map, which includes considerations for people readiness, process improvement and technology-specific implementation plans. Each of these implementation plans should incorporate change management or adoption enablement techniques.
  8. Determine how the existing technical debt will be addressed. Give highest priority to the backlog of technical debt that will most support the goals.
  9. Design metrics including key performance indicators (KPI) and service level agreements (SLA). The use of KPIs and SLAs will enable the ability to tune, track and validate the success of the solution.
  10. Implement! Agile iterations may include detailed design, proof of concept trials, implementation for one viable application environment and team and then deployment to all target application environments and teams using a well thought-out sequence and strategy for onboarding.
  11. Employ continuous feedback. Monitoring the performance of all application environments utilizing the established KPIs and SLAs will provide unbiased measurement of the system performance.
  12. Employ continuous improvement. Feedback is critical for learning and improvement which will fuel each step in your ongoing DevOps journey.
Summary

What do you think? Do you agree the Total DevOps steps listed above are important to achieving big picture DevOps goals? Are there others you recommend? For more information about Trace3’s Total DevOps transformation approach, refer to our perspective paper, “Navigating Your Total DevOps Journey.”

While DevOps implemented using the Total DevOps approach provides a strong foundation for long-term enterprise business improvements, it is important to understand DevOps is not an island. Enterprises implementing DevOps should be aware that DevOps interoperates with other IT systems and practices. Enterprises are well-advised to choose tool-agnostic IT partners that can provide solutions that best suit the needs of each unique enterprise and can integrate and evolve DevOps together with all their IT systems.

— Marc Hornbeek
August 18, 2017
Patrick McClory on Building Successful DevOps Organizations

Patrick McClory has been writing code, managing DevOps and designing scalable application infrastructures for more than 10 years. Most recently, as COO and CTO of DualSpark, he served as an expert Amazon Web Services consultant, helping clients capitalize on the benefits of cloud architecture and design with modern and innovative software, infrastructure and automation strategies leveraging AWS. After DualSpark’s recent acquisition by Datapipe, McClory assumed the role of SVP of Platform Engineering and Delivery Services, where his expertise is put to good use overseeing Datapipe’s global automation and DevOps, as well as its strategic consulting and platform teams. In this role, he manages several teams, including Datapipe’s client-focused service delivery team focused on enabling customers to transform their businesses by leveraging cloud-based patterns and practices.

devopsy.kinsta.cloud: Tell us a little about your cloud experience, your consulting background and your role with Datapipe?

McClory: I found myself in the Amazon partner channel probably a little over eight years ago. I worked for a couple of Amazon partners, and then I eventually went to work for Amazon. After having helped lead some of their consulting work in their professional services organization, which was fairly fledgling at the time, I left to form my own organization. It proved to be an interesting divergence for me. Where I previously spent a lot of time helping clients solve technical cloud problems—which is typically technical and very straightforward—I found myself transitioning to the greater software- and people-problem challenges around effectively taking advantage of cloud technologies.

That’s really how DualSpark was born, both in terms of cloud data architecture, design and consulting. We helped customers understand how to take advantage of the cloud from a business perspective, as well as dive all the way down into the deep, nitty-gritty of exactly how they should technically do it. After eight years of consulting in that world and Datapipe acquiring DualSpark, I found myself in an interesting place where I now run the consultative, the engineering, the client-face role for Datapipe. I also run our platform-engineering team. The big story behind that is trying to take all the knowledge and all the patterns and all of the collective wisdom that we’ve gained over the years, between myself and the team, and building product to help our customers get there faster.

devopsy.kinsta.cloud: We’re many years into the movement toward DevOps. What do you think we’ve learned the biggest challenges to be when it comes to getting DevOps right?

McClory: What we’ve learned is that the technical side of the argument, when it comes to actually achieving a DevOps workflow, is probably the least complicated. The real challenge is getting people aligned and focusing the effort in the right direction. For me, that’s been the challenge. But people gravitate toward the tools because they’re easy. Realistically, you can’t take advantage of the tools if you don’t get yourself organized and going in the right direction.

I’m going to tell a story. There was probably a span of six months where I had the, “What continuous integration tool should I use?” discussion. Invariably, every time, everyone ends up with the same sets of tools. Everybody ends up with Jenkins and, realistically, no one’s always terribly happy with it. Still, it was a six-week conversation, no matter what they eventually chose. Then, rather than starting from nothing, I decided to come to these meetings with a pre-configured system. I would load it into their AWS account, or wherever made sense. They’d see Jenkins hooked up, fully, to their source control toolset. It was highly available and had all of the plug-ins they sought. We took away everything to discuss. We’d then take the next two hours to configure it, play around with the bells and whistles and knobs. Fit it to their purpose. They could always shift to something else down the line.

We couldn’t believe it, but that tactic worked. I no longer had six-week discussions about CI and what tool to use. I had a two-hour, fit-for-purpose discussion. At that point, bells going off in my head, going this is very interesting. What if we could do this more deeply? What if we did this all the way through the entire stack? What if I gave people multiple options and we just went through and picked it, like a menu? I’m not going to say that’s the overall product design. It’s a little bit more complicated and more intricate than that. The whole goal is to help customers get to the transformative place where they want to be, faster, by avoiding some of the common pitfalls of technologists who like to chase shiny things.

devopsy.kinsta.cloud: With that in mind, what are some of the biggest challenges when it comes to getting the processes and people in place?

McClory: IT as an organizational concept within companies that hasn’t done themselves any favors in the last 20 years. There are several different takes on this, but all basically converge on the idea that since the IT service management boom, IT as a discipline hasn’t really matured. Since then, there have been many, many inflection points and/or maturity moments when IT can provide services. But there’s this awesome gap that continues to grow between what the typical IT organization can deliver and what the business is demanding. That’s where shadow IT shows up. It’s like to divergent lines and that gap is shadow IT.

What’s happened is that IT never, and I say this with the internal acknowledgment that I’m a part of this community as well, really done a great job of connecting our mission to the value of the business. The cost center, the revenue center, there’s all these ways to look at it but at the end of the day, IT is a burden, not an enabler. That’s why a lot of these technologies in DevOps are such a focus. The pain is so great that organizations don’t want to fix the systemic problem because they need to stop the bleeding. Often, what we’ve found is that there’s a combination approach that’s really necessary to build trust within that relationship.

devopsy.kinsta.cloud: What do you think IT teams must do to flip IT from that unhelpful department to become more aligned with business objectives and profitability? Not that IT must be a profit center, but be more aligned with the business.

McClory: Being more aligned with delivering value, right? I think it’s two vectors. There’s the one half that’s easy and stereotypical and that’s just because that discipline hasn’t really aligned that way for so long. As silly as it sounds, I find that there’s so much animosity between business and technology organizations on the stereotypical kind of day. There’s a little bit of corporate therapist that we play and we help build bridges, help to realign, help to translate. It’s a surprisingly entrenched issue of people at an interpersonal level.

It’s not simply between the technologist and the company and the business. It’s also even within the technology organization, between your typical developers and operations organizations. Getting those three views aligned to serving the value of the business is, a lot of times, a matter of helping people to acknowledge that other people are a part of it and that they’re all there to serve the same mission.

On the other side, there’s also this misnomer that there is a silver bullet, from a tooling capabilities perspective. There are some great tools out here and I’ve both built partnerships with them and used their tooling heavily. One that comes to mind, particularly, is Chef. I love their tools. They don’t solve every problem I have, though. There’s a landscape of tooling that I tend to end up back at that standard configuration management, continuous integration, testing and deployment management, monitoring and alerting. You end up finding good options in each one. Not that they’re perfect solutions, but a lot of organizations are looking for that DevOps in a box. They’re looking for that one capability and they forget that it’s not one thing. Even if people have tried to market it that way. It’s very much a technology plus an organizational culture issue. Not a single technology.

devopsy.kinsta.cloud: Where do you see your clients now in the curve of DevOps adoption?

McClory: If you’re looking at a bell curve and cut it into quarters, I think we’re probably in the early phase of the second quarter of adoption. There’s a ton of interest and I think that’s what I would look at, in terms of volume. I see lots of people who’re interested. I don’t remember who said it but to paraphrase, there’s a lot of people talking about it. There are very few people who actually achieve it.

At the same time, I don’t believe that, to your point, DevOps isn’t a destination, it’s a way of doing things. It puts you on a vector, it pushes you in a direction. It’s never about the destination, it’s about the journey.

In that respect, I think there are a lot of DevOps practices that start to seep in at the cultural level. As organizations try to be more efficient, we see them coming into some of the automation practices and the tooling side of things much more eagerly than they are the organizational collaboration side. Even that’s starting to break down and, oddly enough, I don’t know that I would blame it on the influx of newer developers or younger developers into the market. Certainly, it plays a role as the—not to wax poetic—the culture of sharing economy, the open visibility outcry from the hipsters perspective, or the millennial view. I think a lot of that I really positively impacted organizations behind the scenes, to open up a lot of that collaboration and visibility that’s necessary to get DevOps right.

— George V. Hulme

July 6, 2017
In DevOps, Figure Out Why Before Considering How

It has been fairly well-established that DevOps is primarily a function of the underlying culture—about breaking down barriers and shifting roles so that teams and individuals are empowered to come up with innovative ideas and execute them without the burden of traditional corporate bureaucracy. It is also understood that there are a variety of tools and platforms that enable and automate processes. However, before an organization can even get to the “How”—the culture and tools of DevOps—it should be able to answer the question, “Why?”

I reached out to organizations that have embraced DevOps and leaders in the field to see what they had to say about the “why.” What benefits can businesses expect, and what goals can adopting DevOps help them achieve—or achieve faster, or achieve cheaper?

“Delay in delivering innovation comes with an increasingly high price tag, perhaps the loss of an entire market to competitors,” says Forrester Analyst Charles Betz. “DevOps accelerates and improves the quality of software delivery, but that is not an end in itself. The increased speed provided by DevOps is essential to achieving the fast product feedback needed for digitally transforming enterprises to stay relevant and competitive.”

As Betz points out, companies should implement DevOps for the competitive advantage. Dave Lindquist, IBM Fellow and VP of Private Cloud for IBM, puts it another way: “DevOps is at the heart of the digital transformation. It’s how the next wave of disruptions are being developed and delivered.”

DevOps bridges the gap between development and operations teams. Inspired by agile development and continuous integration, and incorporating their principles, it addresses the challenges of today’s condensed timeline for application development.

In that way, the line between the “Why” and the “How” blurs into a bit of a gray area. Companies should adopt DevOps with specific goals in mind—a vision for what they expect it to do for them—because simply eliminating barriers or implementing tools and platforms without a strategy in place for what that is expected to accomplishment is likely to be a waste of time, money and effort. On the other hand, it is very difficult to execute or achieve those goals without also successfully navigating the “How.”

“Enterprises are navigating major technology shifts and demand is therefore higher than ever for simple and consistent automation and management across hybrid infrastructure and application portfolios,” says Marc Holmes, VP of Product and Revenue Marketing at Chef. “Organizations should adopt DevOps practices for both the culture and tooling to create a consistent and collaborative workflow so enterprises can build, manage and deploy software quickly and safely in any environment.”

“DevOps can have a direct effect on your bottom line. According to a recent study from Enterprise Management Associates (EMA), ‘Companies that excel at DevOps interactions are 11 times more likely to see double-digit annual revenue growth. It doesn’t stop at revenue growth either. According to EMA, 49 percent of organizations also agree that accelerated delivery frequency leads to higher customer satisfaction, and 35 percent find that it enables them to be more agile and competitive,’” notes Chris Lippi, VP of product at NGINX.

With its focus on automation, DevOps also helps to ensure predictability, efficiency, accuracy and the standardization of operational, development and testing processes. This, in turn, saves time and money by shortening the release cycle, reducing errors and lessening the need for time-consuming fixes. It also frees up resources to focus on more creative endeavors and on tackling larger problems as they arise.

Lippi sums it up this way: “Once implemented, DevOps tends to prove its business value in terms of time and cost savings. Making the business value of DevOps clear—to the newly unified team and beyond—helps DevOps culture take root.”

There are lots of tools and practices out there to choose from, and plenty of guidance and advice on how to implement a DevOps culture. Before you go down that path, though, step back and figure out what you hope to accomplish by adopting the culture, tools and practices.

— Tony Bradley

June 23, 2017
Avoid Crashes: 5 Best Practices for Load Testing

Demanding customer expectations and the rapid distribution of information across social media have made website and applications crashes an off-limits path for today’s companies and organizations.

Crashes resulting from traffic spikes cause immediate and long-term loss of sales, as well as tarnishing the brand name. When servers or websites go down during highly anticipated events such as Black Friday, the Big Game or before a major concert, the repercussions can be permanent.

Yet, avoiding the “website temporarily unavailable” screen is also easier than ever before. By performance testing your website continuously and proactively you can easily detect errors, bottlenecks and bugs, and then fix them to ensure your website handles any amount of traffic.

Here are five best practices for load testing your website or app:

Define Your Load Goals

Before starting your tests, you need to figure out what it is that you need to test.

This includes the number of concurrent users, traffic patterns and areas that users can visit on your site. For example, if your site has 10,000 users concurrent at its peak, but 3,000 arrive in the morning and that total slowly climbs up to 10,000 in the afternoon—and these visitors mostly just browse your homepage—you’ll need to run a different test than if you have 10,000 concurrent users on your website all day long, constantly browsing multiple pages.

To determine your load goals, you need to engage directly with your business colleagues. For instance, get your product manager to set the system’s expected abilities, ask the marketing team to inform you about any special campaigns that might increase website traffic, and then look at server logs and tools such as Google Analytics to determine previous customer patterns.

Choose Your Load Testing Tools

There are multiple open-source load-testing tools that you can use to run your tests.

JMeter is the most popular tool, but you can also use Gatling, Locust, Tsung, The Grinder or Selenium. See here to learn more about the differences between these toolsets.

Does this seem complicated? Taurus is an open-source automation test framework that simplifies running these tools, by using a simple YAML script. If you need to run your tests from the cloud, utilizing rich analytics or requiring user collaboration, you might consider using BlazeMeter.

Take Your System to the Limit

To make sure you stand up to your load goals, you need a buffer; specifically, you need to make sure that your system can handle heavier loads.

This ensures that the CPU stays low and you don’t experience memory leaks. However, we also recommend you take your system to the limit and see when and how it fails. This ensures that you know exactly what your system’s weaknesses are, giving you a better ability to fix any errors that might occur.

This is especially important if you come across an unexpected traffic spike—say, if your competitor’s website crashes and all their customers come running to you. Wouldn’t you rather be prepared for that?

Monitor Test Results

Set aside resources for looking at test results.

Examine KPIs including Response Time and Hits/s, as well as the correlations between them, to learn what your system’s abilities and bottlenecks are. This information will help you see what needs fixing.

As you continue testing over time, you will be able to identify KPI trends, which will help you monitor system health. We also recommend you monitor backend KPIs like DB Queries, with APM tools.

Test Again

So, you’ve tested, you’ve analyzed, and you’ve fixed.

Now you have one last thing to do: Run your tests again to make sure everything is working smoothly after the committed changes. That ensures nothing gets overlooked. It would be a shame if all your hard work went down the drain due to a minor oversight.

Now, let’s get testing!

To get more load testing tips, check out this free whitepaper, “How to Ensure Your Website or App Won’t Fail at Peak Times.” To learn more about BlazeMeter, request a demo.

— Noga Cohen

May 16, 2017
DevOps is a Lifeline to Connected Mainframe ROI
A new IDC white paper sponsored by IBM and CA Technologies reveals the adoption of DevOps is a hallmark of those organizations that are actively modernizing and integrating legacy systems such as a mainframe to support digital transformation.

Early adopters are blazing a path for faster innovation and greater ROI.

DevOps Delivers Mainframe Modernization and Integration

I’m encouraged to see DevOps recognized as an important element in realizing a “connected mainframe” vision. Respondents demonstrated a continued commitment to modernizing their mainframe environment.

The IDC study confirmed that leveraging Java on the mainframe was one of the most popular approaches. In fact, I’m constantly surprised at how few managers realize that these days, Java and Git can be used by developers in such a way that allows them to completely separate themselves from the mainframe “green screen” environment when coding or maintaining applications.

DevOps automation has also helped organizations overcome many of the cultural and operational challenges to a fully end-to-end enterprise by aligning application development tools and orchestration chains across multiple platforms and teams.

Adopt a Business-First Approach

Customers I’ve spoken with are experimenting with DevOps for mainframe tools and practices, with many piloting new approaches across the life cycle. They’re immediately becoming more agile, realizing shortened development cycles, improved collaboration between distributed and mainframe teams and faster MTTR that maximizes availability. In all, these early adopters have benefited from increased speed to market, improved business agility and better platform economics.

The IDC study concurs, identifying more effective delivery of applications and services as a powerful cost-saver, with savings of $58,900 per application achieved through increased developer productivity and day-to-day support efficiencies. By helping mainframe organizations to realize improvements at every stage of the application life cycle—from build through test, deploy and operate—DevOps delivers compelling benefits; for example, from our recent study Accelerating Velocity and Customer Value with Agile and DevOps:
- Advanced DevOps users reduce the time taken to develop, test and release new applications by 42 percent.
- Advanced agile users reduce their time to decide and act on new opportunities by 40 percent.
- Advanced agile users accelerate the time to develop, test and release new apps by 36 percent.¹
Excited? I encourage you to catch up on the full IDC White Paper here.

— Aline Gerew
May 3, 2017