Tag: devops culture

Your DevOps Subculture

I’m often asked how to create a DevOps Culture and there is no single answer to this question. The focus initially for me in any cultural movement is to understand how the organization incentivizes and what its core beliefs are. When it comes to DevOps, I am normally looking for ways to bring these two teams together from a fundamental aspect first. One method I have found that works well when I’m consulting with these two teams is a Culture Session.

Teams often end up with a subculture that incorporates aspects of the macro culture. These subcultures can evolve into the silos that DevOps is designed to break down. A Culture Session is one where you bring these teams together en masse and work on defining a new unified subculture.

Culture Session

The session starts before we even meet, with buy-in from the leadership. There will likely be detractors that will not want to attend or contribute. Having leadership not only broadcast the meeting, but support the change will be critical to success. We want to address this issue from the top down as well as from the bottom up.

With both teams in the same room we will begin to address the culture we want. Language should focus on the fact that the team is made up of all engineers. Creating a sense of a single team spanning Development and Operations is a key focal point in DevOps. In order to build a unified new culture, we must discover the core beliefs of the new team. Ask the team what they consider our core beliefs of their culture is. Don’t restrict the statements that come out of this part of the session, everything has merit and should be included.

It can be difficult sometimes to get a team to think in terms of beliefs. If we are greeted with blank stares or eye-rolls when asked what we believe in, try a different tactic. Ask the team what would be an ideal company to work for and build beliefs off of these statements. Here are some examples from my sessions, converted into beliefs.

I want to work for a company that…	Belief
is fun to work for	We value a fun work environment
let’s me work from home	We value results over where work is accomplished
wants to hear my ideas	We value everyone’s input always
helps me to grow my career	We value the growth of people as much as the business
has a diverse workforce	We value equality in all aspects of our culture
has a community that supports each other	We value consistent adherence

Once you have a fair number of items (based on the number of participants) we can begin to order the list. It’s acceptable to write these on yellow sticky notes as well if you don’t have a whiteboard. The ordering is accomplished via a process similar to voting in Lean Coffee. Everyone gets three votes; dispersed any way they desire. I tend to use dots on the cards and then add up each card. List the top 5-10 cards from highest to lowest, and these are your top cultural values. Discuss the results with the team, are they surprising?

Aligned Goals

I believe that aligned goals create a more comfortable work environment and encourage the building of trust. Your teams goals should not only be aligned with the business, but also with your culture. This will make introducing and reinforcing the proper behaviors easier. Some of the highest performing teams I’ve worked with had alignment on all three fronts: Business, Behaviors and Culture.

Behavior and Community

Now that you have a core set of beliefs, it is time to begin talking about the behaviors we wish to see that drive those beliefs. The leaders at all levels must continuously encourage the community to reduce behaviors that are not aligned with core beliefs, while encouraging those that do. Over time the community will govern its own members, solidifying the positive culture and beliefs. The ability of the community to manage its own members requires the alignment I spoke of earlier.

DevOps Language

What I call DevOps Language is key to building and maintaining a strong community. A major component to DevOps is a culture that is accepting, open, honest, collaborative and trusting. We tend to get defensive when we feel that our abilities or intentions are being called into question. The following shifts may appear simple and non-significant in the beginning, but these simple changes in how we communicate greatly enhance our ability to clearly get our message across.

Why to How

Moving from “why” to “how” is a great first step in adopting a DevOps Language. Often we are faced with a lack of understanding that we desire to fill. Perhaps you are looking to get a better handle on the design or structure of a block of code or infrastructure. Maybe you are attempting to learn a process on a new team you’ve just joined. Regardless of the scenario, you are going to have questions. Like many “trigger” words, “why” tends to send some people into defense mode. Why can (and tone of voice is crucial here) make people feel as though you are questioning their intelligence and intent or even implying that you would have done it different (aka better). Moving the language from why to how you did something invites the listener to share their reasoning and experiences, not defend their position.

I Know to I Agree

Another simple change that I have found and adopted into my DevOps Language is “I agree”. I had a bad habit of saying “I know” when someone would tell me something I… well that I already knew! Again, the issue here is subtle, but with all things subtle they build over time. Saying “I know” sounds condescending at times and can be seen as an attempt to exert your superiority. In other words, this is not an collaborate language tactic. When we say “I agree” instead, we are saying that we are aligned in our thinking. This is including all parties and doesn’t shut the conversation down.

Should’ve to Will

DevOps is all about collaborating to make our future better. We often need to review events of the past to identify how to create a better future. The key to this review is to not focus on counter factuals rather we use forward thinking language. Concentrate on moving from “we should or could have” to “we will”. Over time this will help the team to maintain a forward thinking bias rather than dwelling too long on the past.

Closing

Creating systemic long term positive change within your organization takes time, effort and alignment. Creating a unified culture and belief structure through collaboration and inclusion paves the way for your DevOps future. Focus on the language and behaviors that reinforce your culture. Encourage members of the community to help govern and guide the culture, using the agreed upon beliefs. Include the beliefs and DevOps language in the interview and hiring process, in order to include new employees early and often.

Let’s collaborate! I am always looking for ways to learn and grow. How have you aligned your teams around culture? How can I make this exercise more effective?

November 16, 2015

DevOps success is everyone’s responsibility

For organizations with existing hierarchies and processes the shift to a DevOps mindset can be a challenge. The traditional departmental roles and separation of duties is anathema to an efficient DevOps culture. When it comes to DevOps there’s no room for finger-pointing—every team and individual is equally responsible for making sure everything is firing on all cylinders.

Gene Kim, coauthor of The Phoenix Project: A Novel About IT, DevOps, and Helping Your Business Win and the upcoming DevOps Cookbook, took part in a podcast earlier this year and shared that two-thirds of the DevOps success stories he has seen are actually from established businesses with existing infrastructures and processes as opposed to “greenfield” projects that embrace DevOps from the start. In those situations, the drive to embrace DevOps has to come from somewhere, but in order for DevOps to work the organization has to overcome the urge to place the entire burden for the success or failure of DevOps on that one team or individual.

There is a reason that the word “culture” plays such a central role in any discussion of DevOps. Without a shift in the mindset of how teams and individuals communicate and collaborate a DevOps initiative isn’t much different than any other IT project in the organization. One team or person can use DevOps tools, but that’s not really a functional implementation of DevOps.

That doesn’t mean that DevOps is an all-or-nothing game. It’s possible for different departments or elements within a company to embrace DevOps without the whole company jumping on board. There has to be a quorum of sorts, though—a broad enough consensus between teams and individuals to cooperate and collaborate in a different way than how it’s laid out on the company org chart. An important part of that shift is for everyone to accept responsibility for the outcome and work together toward a common goal.

There’s a phrase that goes something like “When all you have is a hammer, everything looks like a nail.” Developers understandably view the world through the filter of developing. IT operations people look at projects from the perspective of operations. QA looks at things from the point of view of testing and quality assurance. Accounting considers projects from the financial angle. Security personnel tend to focus on the security aspects. You get the idea.

Kim shared an excerpt of the upcoming DevOps Cookbook with me. In the book Kim talks about the need to move from functional-oriented teams optimized for cost to market-oriented teams optimized for speed. He talks about the challenges and bottlenecks created when some teams or elements are moving at an accelerated pace while others struggle to keep up.

Part of optimizing for speed is shifting the perception of responsibility. Smaller, market-oriented teams are more agile and can deliver value to customers—both internal and external—while reducing or even eliminating reliance on other teams. Kim explains, “Taken to its extreme, market-oriented teams are responsible for not just feature development, but also testing, securing and deploying their code into production themselves, as well as being responsible production operations and availability.”

In order for DevOps to work effectively, everyone needs to forget that they’re a “hammer” and view the project as a more comprehensive toolbox instead. DevOps teams need to strive to break out of the traditional roles and silos and make a conscious effort to see things in terms of the common goals or greater good.

October 20, 2015
Dice Breakers: using DevOps principles and nerdery to reimagine Team building
Software is a young industry, and not a very traditional one. Your team can’t always be expected to behave like teams in other fields.

So it’s no surprise that when a development team hears the phrase, “teambuilding exercises” for most of them it conjures up the idea of a room full of awkward people marginally engaged in a trite set of activities.

But given the resonance of ideas like DevOps it is apparent that the benefits of teambuilding are still sorely needed. Software companies can fall into a very ‘silo’ed mentality. This is often to our detriment, and tearing down those barriers is a big part of a DevOps philosophy.

So there is an impasse; prepackaged, standardized teambuilding exercises fall short of a development team’s needs, and the need for the lessons those exercises try to get across is unmet.

Last year, when I was working on Release! the Game, I realized something; the things we do outside of work, still resonate inside of work. That led me to think that Office Game Nights might be an interesting, albeit nerdy, solution to this issue. But before I explain my dice driven solution, let’s talk a little about traditional team building.

Classic Teambuilding

For the sake of discussion let’s say that by and large teambuilding activities all have a similar set of goals:
- Encourage communication
- Create easy references for actual on the job problem solving
- Relieve stress
- Provides a structure for complaints and socializing
How they try to accomplish this varies, but as a general rule they set forward an infrastructure for interaction that is easily explained, and with a mechanic for comparison to the real work day.

For example, in the classic Trust Fall exercise, one person is called out, to fall backwards while their team is called to catch them. Takes a minute to explain, and the concept that we have to rely on and trust our coworkers is applicable almost everywhere.

It’s a fairly simple formula, get the whole office together, present a challenge, collaborate to meet it, and then discuss the vagaries of how you worked together. Your team bonds a bit, and gets some reinforcement on general work principles.

Why doesn’t that work for Software

In the case of software development teams we actually wind up seeing these results:
- Sighs, scoffing, and somewhere in the dozens of eye rolls, depending on team size
- Some variation on the phrase, “I could be getting real work done now.”
- Some variation on the phrase, “At least I’m not doing real work right now.”
- One or two sincerely interested participants
Why?

Whatever your place on a development team your job will revolve around creating or manipulating complex logic chains to overcome some hurdle. This, in and of itself, is not that much different from any other job, the difference is code.

Code is abstracted, it is just the logic. Code doesn’t need a teambuilding exercise it just needs to be formed correctly, tested reliably and it just works. If it has a problem, communication or otherwise, there is a distinct and detectable source of that issue, and one that, most of the time, has a methodical solution.

That’s a simplified explanation, but the key there is that there is an identifiable problem, and an identifiable solution. That just isn’t the case for the social systems that operate outside of the code. Navigating the personalities and relationships that form in a workplace simply doesn’t have the same amount of landmarks that navigating code does.

The pit fall is in thinking that the software you produce, and your company are the same creature, or at least have the same needs.

Teambuilding isn’t about the product. It’s about establishing a better system for the people who work on that software, and helping them make the connections that will make production run smoother.

Teambuilding and DevOps

DevOps principles demand strong communication between the members of your team. The tools, the methodologies, all need the right environment to be applied in.

So if you want to build a team that can really talk to each other, you need to step away from the standardized techniques, and pick up some new tools.

For team building ideologies to function they require a sincere investment on the part of the participants, but, in most cases, they are so pre-packaged that the exercises themselves lack sincerity.

Openly acknowledge the goals that your teambuilding program is trying to achieve and don’t couch it in the window dressing of corporate training. There are plenty of structured activities that get folks talking, let them blow off some steam, and foster a sense of teamwork that can be easily reflected during your regularly scheduled workday.

My Answer?

(spoiler alert: this is where it gets nerdier)

Board games: sweet dice rolling, card shuffling goodness.

Game nights (or days) are a great place to do some socializing, have a necessary structure around that socializing, and encourage creative problem solving either through cooperation, or competition. There are also a lot of them. The landscape of modern tabletop nerdery isn’t just defined by Napoleonic conflict, or real estate monopolies, there really is something for everyone.

Setting up some gaming is pretty simple. Set aside a few hours, pick out a game or two, pick up and pizza or two, and get a few coworkers on board. Don’t worry if everyone doesn’t jump on board, give it a few iterations to build steam.

Here you can take a lesson from a DevOps release strategy, if your first night didn’t achieve your results, try again with some small updates, a new game, different toppings, etc…

Chances are someone in your office already is into games, especially in the software fields. But, if not, don’t worry it’s pretty easy to get nerds on the hook.
October 8, 2015
Native Integration of APM Driving DevOps Maturity

With applications performance monitoring established as one of the fundamental cornerstones of today’s DevOps methodologies, it’s easy to overlook the reality that effective instrumentation of such tools remains extremely difficult. Specifically, the biggest challenge remains creating performance metrics that address both IT and larger business initiatives.

There’s no question that APM is one of the most widely adopted and deeply embedded elements in today’s DevOps world, serving as the critical source of measurement that enables more transparent and agile applications. Yet, despite the fact that performance monitoring solutions, and their respective providers, are considered to be among the most mature on the DevOps landscape, customers still struggle to optimize the systems to meet their specific requirements.

However, as the market continues to advance, specifically via direct integration of widely-adopted APM tools with other strategic applications management systems, such as e-commerce platforms, organizations are finding a faster path to success, and bigger-picture DevOps benefits. One such partnership, announced in late August, saw market leader New Relic expanding integration with e-commerce platform provider Magento, owned by eBay.

Jon Azrielant, director of marketing at online retailer Jewelry.com, a Magento customer, said that his company initially ran into significant obstacles in adapting New Relic’s APM capabilities to its processes, despite the comparative strength of the tools.

“We had a really difficult time implementing New Relic initially, it was a painful process,” Azrielant said.

While the APM tools provided Jewelry.com with the data necessary to generate vital alerts, along with more agile sysadmin and operations monitoring, the more silo-based, fragmented approach that it employed could not empower the retailer to circumvent traditional walls between apps support and service support teams, Azrielant said.

Leveraging the extension of New Relic’s Software Analytics Cloud within Magento, the company has created more advanced instrumentation to inform critical business decisions. Among the most helpful of these measurements are those relating directly to customer experiences, while correlation of such metrics with Jewelry.com’s bottom line has also advanced executive decision-making, he said.

“The critical result of this integration is more complete visibility across applications and business performance, importantly tied to the actual business ramifications,” said Doug McIver, director of product for Magento. “In the past, merchants had to do a lot of difficult integration to use these systems together, so obviously this is a hugely valuable capability for us to be able to offer, especially to customers with highly performant site requirements.”

Moreover, Magento claims that it is growing its own customer base by tying APM more closely to its platform. Likewise, New Relic officials concede that to help advance crucial DevOps workflows, it must partner with such platforms to deliver APM data within the context of business requirements.

“If we’re talking about how to win in world where we’re iterating the end users’ experience on a regular basis, we have to put performance metrics within context of the UI, and the business results; working from the same data sets and getting people on the same page is incredibly important,” said Abner Germanow, senior director of solutions marketing at New Relic.

While pure performance stats are “near and dear” to New Relic for obvious reasons, the solutions provider is also attempting to advance a broader array of information that drives closer collaboration through new “iteration and experimentation”. New Relic is also focused on further developing the specific types of information most critical to key constituents such as e-commerce platform users. The need to create a common language for those DevOps efforts is amplified by the fact that development and operations officials are often unsure which metrics are critical to the business, and vice versa.

“Instrumenting code has always been a challenge, and we’ve worked very hard to ease that pain across a lot of different platforms; there’s always another language where work can be done, and we look to our customers to help lead us into the next phase of instrumentation,” Germanow said. “We’re seeing a lot of pressure around the need for constant change and iteration in industries where it’s truly a matter of survival, such as e-commerce, publishing, digital governance and finance – places where the industry is behaving very differently now than it has over the last 50 years.”

This work, and the subsequent APM integrations offered via platform providers such as Magento, have allowed end user organizations to remain focused on customer experience and sales, the ultimate goal of DevOps, said Jewelry.com’s Azrielant.

“In terms of how this relates to the DevOps revolution, this is the advancement that is bringing everything together and allowing everyone to look at the same numbers, to share common APIs, to streamline reporting and event response, and define everything in uniform terms,” Azrielant said. “Ostensibly, the most basic tenet of DevOps centers around the unification of these disciplines, and you can’t support that without the unification of this information.”

September 29, 2015
4 big challenges of microservices

The rise of microservices as a means of developing and delivering software value faster is clearly on the rise. Containers have skyrocketed from a fringe technology to a mainstream imperative and organizations are scrambling to jump on the microservices bandwagon. Before you rush out to embrace microservices, though, there are a few things you should keep in mind.

First let’s talk about what microservices even are since it’s a relatively new term in a sea of relatively new terms.

Al Hilwa, research director, Application Development Software at IDC described microservices this way:

“Microservices is an architectural approach that draws on long evolving experience in software engineering and system design, including the SOA efforts of the last two decades. Microservices architecture is enabled by a spectrum of tool categories but is primarily an architectural approach to system design that also requires considerable organizational and cultural adjustment to execute successfully.”

Put more succinctly, microservices is an evolution of software development and deployment that embraces DevOps and containers and breaks applications down to smaller individual components. Organizations can develop and deploy faster because there is a modular approach that makes code more efficient and enables much of the common or routine elements to be automated.

Everyone seems to be doing DevOps and containers these days. If you’re not you may feel like you need to catch up quick or risk being left in your competitors’ collective dust. That may be true to some extent, but you have to approach microservices in the right way and for the right reasons. Jinesh Parekh, CEO of Idyllic Software, shared some thoughts about four challenges organizations should consider when venturing into the world of microservices.

1. Insufficient guidance

When it comes to implementing new technologies and embracing new strategies it can be a little like the “Wild West”. There are few hard rules in place and very little in the way of guidelines or documentation to help you navigate how to build and deploy software using microservices. Thankfully, sites like devopsy.kinsta.cloud and ContainerJournal are here to provide as much information as possible to give organizations some guidance.

2. Getting it wrong

There are a variety of “moving parts” involved in doing microservices successfully. Your organization should adopt a DevOps culture and have the appropriate infrastructure in place to avoid getting it wrong. Microservices architecture is generally built on a foundation of cloud and virtualization, and relies heavily on automation to ensure success and avoid failures.

3. No going back

You should be aware that it’s exceptionally difficult to switch back once you start down the microservices path. The changes that have to happen in terms of culture, tools, and processes to embrace microservices are not easily undone. The good news is that there is very little—if any—reason an organization would move to DevOps and containers as a development framework and then decide to revert to the old, monolithic way of doing things.

4. Deliver value

Parekh’s focus on delivering value is biased toward Idyllic Software. His company is in the business of delivering applications for clients via microservices and Parekh suggests that organizations outsource development to an experienced leader to ensure microservices meets expectations. You don’t necessarily have to outsource to Idyllic Software, though, or outsource at all. You just need to make sure that your approach to microservices benefits your organization—that it adds value and meshes with business objectives and isn’t just microservices for the sake of following the microservices trend.

Microservices seems to be here to stay, and seems to be on the same path as DevOps and containers from niche concept to mainstream business adoption. As you ramp up to jump on the microservices bandwagon just make sure you keep these things in mind.

September 21, 2015
Librato Seeks to Unlock DevOps Application Monitoring

As wider adoption of DevOps methodologies continues to foster demand for new tooling to help streamline evolving workflows, at least one vendor is attempting to address the growing need for related monitoring capabilities.

Librato, a San Francisco-based provider of Amazon AWS applications performance monitoring (APM) services, claims that it is finding favor among the rapidly expanding DevOps community driven by its ability to inject critical visibility into the engineering lifecycle.

Acquired by Austin, Texas-based SolarWinds in January for a lofty $40 million and renamed Librato SolarWinds Cloud, the company is touting its recent adoption by well-known providers such as Slack for monitoring of DevOps-centric applications performance and health metrics.

“At its core, DevOps is destroying conventional wisdom around ritualized processes and the need to perform them, mostly in the domain of scale,” said Dave Josephsen, Developer Evangelist at Librato. “By attempting to include monitoring as a key element of the development lifecycle, this has created a whole new range of challenges, and opportunities.”

By providing a centralized Web interface and open API to help create visibility into critical applications engineering requirements, Librato maintains that it is creating the tooling necessary to empower integrated development and deployment processes, such as continuous change control.

“There are a lot of disparate tools for monitoring individual engineering tasks, but there hasn’t been any practical way to bubble up that data for efficient monitoring,” Josephsen said. “Within the context of DevOps, change control is a ritual that we’ve abandoned as we’ve embraced test driven development and automated away the human element, so there’s a distinct need for monitoring of feedback and metrics.”

For example, cloud-based productivity messaging specialists Slack are using the Librato “StatsD” lightweight daemon process to stream business and app-health metrics regarding PHP applications into the monitoring platform. Slack also leverages Librato to chart data regarding connect, query and fetch latencies for its server clusters, and to monitor internal metrics about overall system health.

“Having those graphs side-by-side lets us clearly see and compare the effects of network latency on one of our new services, and that’s very cool,” Richard Crowley, director of operations at Slack, wrote in a recent testimonial posted to the Librato Solarwinds blog.

Industry experts agree that broader adoption of DevOps strategies will continue to drive demand for relevant monitoring tools, with solutions such as Librato seeking to deliver “modern stack architecture for looking at the modern stack.”

“I think that APM is always interesting because it’s always new,” said James Governor, industry analyst with RedMonk. “The pace with which we’re building new architectures to deliver services to end users is increasing all the time, and with every new wave of technology, we tend to get new and associated models of APM tooling.”

Librato is taking advantage of the need for a system that brings together the elements necessary for cloud native monitoring, and in an agentless manner different from other competing tools such as New Relic, the analyst noted.

However, whether or not the platform is able to break down traditional engineering siloes to enable more efficient DevOps management, as Librato officials claim, remains to be seen, he said.

“Each wave of technology has promised to do this, to remove the inherent finger pointing, and each has always made the same claim to break down organizational firewalls,” Governor said. “DevOps hasn’t boiled the ocean in this manner yet, but there are some unique opportunities; unlike previous waves, where automation was intended to replace humans, this is now about augmenting people, and we are seeing that teams are choosing new tools from which developers can truly derive value.”

August 10, 2015
The Art of DevOps
This is the third in a series of posts on DevOps. The first written by my colleague Lee Reid was titled The Simple Math of DevOps. The second The Calculus of DevOps was written by IBM client, and my friend Carmen DeArdo of Nationwide. Lee introduced mathematically improving delivery throughput by leveraging DevOps to improve Trust. And Carmen used Calculus to imply that the business value of IT can be improved by adopting DevOps to increase the frequency of releases. In my post I would like to visit the art of DevOps , or the art of adopting DevOps by picking up right where Carmen left off – by looking at the culture side of adopting DevOps.

Lee and Carmen have already alluded to some basic premises behind DevOps as an approach – reducing batch sizeand improving collaboration. These result in reducing complexity, increasing throughput, and improving trust. Let’s talk about how to get there.

As I have worked with customers adopting DevOps, across company sizes, across industries and countries, there are four common threads that make DevOps adoption successful, leveraging all three aspects of DevOps adoption – culture, process and automation. These are:
1. Reduce batch size: The most effective way to manage risk and quality, while increasing speed is to reduce the batch size in each iteration or ‘sprint’. This is a mind shift to release smaller, more frequent new versions. Where it is not possible to release new versions that frequently to customers, release small batches to a pre-production area. The deliverable in pre-production is tested and made customer-ready, and then released to the customer at formal release dates. As Lee and Carmen have both discussed, reducing batch size is a pre-requisite to improving both trust and business value.
2. Shift-left Operational engagement: Get operations engaged right from the beginning of a project/initiative. This has been a core principle of DevOps. Do not leave Ops in a silo that one throws code over the wall to, but have them engaged in the development process, right from requirement inception stage. This allows Ops to have visibility into what is coming from Dev; allowing them to deliver the right production-like environments, as and when needed to Dev and Test teams. It also allows Ops to shift operational concerns left, left into the minds of developers, making them more Ops conscious and engaged in what happens after they hand-off to Ops.
3. Continuous funding: This is a significant change in an organizations funding model to enable continuous availability and engagement of SMEs and business functions, previously not engaged on a continuous basis – like security, legal, marketing, etc. Carmen in his post mentioned several examples of ‘waiting for…’ work, someone else to do something and environments. Another ‘waiting for’ in enterprises is for funding. Funding is typically provided in a waterfall manner, with specific hard dates (Months, quarters, fiscal year) and gates, not suitable for a Continuous Delivery model. Funding too should be continuous.
4. Create a ‘Product Management’ team: This team includes a Designer, Development lead, operations lead and an Architect at the minimum (4-in-box). They own the ‘product’ through its entire lifecycle, and beyond transient projects. They are responsible for long-term thinking, design thinking, evolutionary architectural thinking, and overall ownership of the product, from concept to end-of-life.
5. Setting up a DevOps Center of Excellence (CoE): This center of excellence is not an administrative organization or a ‘tools/enablement group’, but a place where DevOps adoptees come to learn from each other and share expertise and lessons learnt. As organizations adopt and scale DevOps adoption, this CoE also becomes a source of DevOps coaches, helping teams and programs adopt DevOps, and own the organizations DevOps framework – their own flavor of DevOps.
Adopting DevOps is an art. It is not as simple as hiring a consultant who can show how to improve processes; it is not just buying and adopting tools to automate manual tasks; and it is not going through ‘fall-back-into-the-arms-of-the-person-behind-you’ exercises to build trust. It is a combination of all three – process improvement, organizational and cultural change, and automation with tools to replace manual processes. Adopting these in any enterprise, outside of startups, requires overcoming organizational inertia. By changing culture not by edict, but by showing the need and value of change. By building trust, not through meetings, but with improved communication and transparency. By improving business value, not by measuring mandated KPIs, but by improving organizational agility and throughput. Overcoming organizational inertia, by delivering towards a common goal for the business, irrespective of where in the organization one is, and what role one has. By developing a culture where everyone takes responsibility of delivering value to the business.

Watch the webcast series hosted by devopsy.kinsta.cloud in partnership with IBM or download a copy of DevOps for Dummies book for more information.

About the Author/ Sanjeev Sharma

Sanjeev Sharma, CTO and Distinguished Engineer – DevOps Technical Sales and Adoption, IBM Cloud Unit, is a 20-year veteran of the software industry with expertise in DevOps, Mobile Development and UX, Lean and Agile Transformation, Application Lifecycle Management and Software Supply Chains. He is a DevOps Thought Leader at IBM and speaks regularly at conferences. He has written several papers and is the author of the DevOps for Dummies book.

Contact Sanjeev on
August 6, 2015
#HugOps in Practice: Empathy Skills for DevOps
Get it together! Why is your service always down?!”

We think we’re doing the whole DevOps thing right — new hires can deploy on day one, Travis CI is humming along, and we own the code we ship. But then something breaks, something doesn’t go according to plan, tempers flare up, and all that warm, fuzzy collaboration seems to evaporate. What’s going on? What happened to #HugOps?

A large part of software engineering doesn’t involve code at all— it’s talking and collaborating with our teammates. Soft skills are harder to measure and quantify than performance or reliability, but they’re just as important— and while we’ll readily spend hundreds to thousands of dollars on books, courses, and training to improve our software skills, companies rarely invest in creating the culture of empathy and compassion that was behind DevOps in the first place.

Empathy works great…until it doesn’t. It’s easy to be compassionate and collaborative when stuff is going great and everyone’s having an easy time, but it’s stressful, high-stakes situations that show you whether you’ve actually built a culture of empathy. Empathy skills for DevOps cannot be stressed enough.

Let’s look at some specific strategies and tactics teams can use to improve organizational empathy.

Learn to Communicate

Conflict mediators, couples therapists, hostage negotiators, policemen, and others in high-conflict situations often use specialized communications frameworks to help de-escalate situations and get parties in conflict talking to each other in a productive way.

I’m a fan of a communications framework called Nonviolent Communication, popular in education (I used to be a schoolteacher), parenting, mediation, as well as peace programs in war zones. Another popular guideline is Crucial Conversations, a more business-focused structure for handling conversations where emotions are charged and stakes are high. There are plenty of books, trainings, and other resources about empathetic communication out there, but they all have some common themes:
- When dealing with a tough situation, separate — explicitly— what happened from how you feel about it. So for the example above, you might say something like “When your service went down for the third time in the past two weeks, I got worried, because our service depends on yours, and we’re trying to hit four nines of uptime this quarter.” Often just the simple step of decomposing reactions from what’s being reacted to defuses the majority of the speaker’s anger— this is used in blameless postmortems to move from “blame and shame” to an honest look at the facts of the situation.
- When somebody shares a feeling, especially if emotions are high, take time to make sure they feel heard and validated…before you reply. That doesn’t mean you agree with them, just that you ensure that they know that you listened and are trying to understand. In our conflict above, this might be something like “It sounds like you’re really frustrated that our service went down, because it prevented your team from hitting your uptime goal.” Most of the time, the other person will instantly soften, relieved to feel heard. Sometimes you’ll get a “Well, duh!”, but that’s far preferable to the other person leaving the conversation feeling unheard and unacknowledged.
- If you have a request of the other person, make it positive, concrete, and explicit. And “Could you not?” doesn’t count. Without a concrete request, it’s hard for the other party to know how to meet your needs or improve the situation, and hard to tell whether that effort has been made. For our situation above, this could be something like “Can you please do a postmortem on the outage, and let me know by the end of next week what you’re going to do to prevent a future outage and anything you need from me to help?”
Again, there are all sorts of frameworks available, and like many tooling choices, it matters less the specific one you use than getting the organization on board with improving everybody’s communication and conflict resolution.

Align on Common Values

Cross-departmental collaboration gets much easier if you’re both trying to do the same thing. That doesn’t mean your day-to-day work has to be the same, but the more of your company that is driving toward the same goals, the easier it is to turn conflict into simple questions about sequencing or prioritization, rather than fundamental directional issues.

LinkedIn, Google, Twitter, and other big tech companies favor the “Objective + Key Result” (OKR) system for this purpose, where a big, ambitious goal and set of results at the company level trickles down into corresponding goals and results for the department and the individual.

We use OKRs here at PagerDuty, but there are plenty of other goal-setting systems that work. The important thing is agreeing on the meta-important stuff in plain language, so that when someone asks you to change your priorities or randomize what you’re working on, you can have a conversation about which of the options best supports the common goal, rather than the much hairier discussion about what the end-goal even is.

Mind Your Medium

Particularly with distributed teams, text-based communication can make tone hard to get across. A request said with a warm smile sounds quite different from one in flat text over email (or to everybody in the company HipChat room).

Where possible, I prefer video chat to text chat— in fact, some of our teams use perpetual Hangouts so that they can all see each others’ pretty faces, and feel like they’re in the same room together. Where that’s not feasible, I like to over communicate: if the tone of a sentence might be ambiguous, I explicitly state what I mean and what I don’t mean. And while it’s a little weird to me that this has become business normal, emoticons also go a long way in communicating feels over the wire.

Build Relationships, Inclusively

Ultimately, all the communications frameworks and shared goals in the world won’t take you to #hugopsland if you don’t have a human relationship with the person you’re trying to work with. An HBR study found that while people generally claim to prefer competence over likability, in practice they behave oppositely — likability is much more important for collaboration than competence.

And while a few people might be naturally gifted with irresistible charisma, for most likability isn’t a static, unchanging thing: it’s a quality built on familiarity and connection. If two people are going to be collaborating on something, get them some face time — not just screen face time— with each other. We have an engineering office in Toronto, and try to make sure new employees quickly get a chance to spend time with the team in the other office as well as their own. At DevOpsDays Minneapolis this year, Katherine Daniels shared the concept of “bootcamping,” where each new Etsy hire spends a period of time embedded in a different team, learning about how they work, what their goals and challenges are, and making meaningful contributions to their codebase.

Plan off-sites, social events, time to chill out and get away from the laptops. But keep in mind that not everybody on your team necessarily drinks alcohol, and some of them might (gasp!) have families. That doesn’t mean that you can’t ever plan a team gathering at a bar or do nighttime activities, just try to be aware of the different constraints, preferences, and physical abilities your team members may have. Providing hangout spaces at work is a simple but often overlooked way to help build relationships while respecting those who may have out-of-work obligations or just really long commutes. And while a shared lunchroom can be a good way to cross-pollinate, it’s also sometimes nice to get out of the building— PagerDuty pairs all new employees up with a “buddy,” and sponsors lunch out in the neighborhood to help them get better acquainted.

Being inclusive of different learning and personality types is important as well. Even if you don’t buy into specific personality tests such as Meyers-Briggs, introverts and extroverts handle relationships and communications differently, and it’s important not to assume that everybody handles information or makes decisions the same way you do. When doing a sprint retrospective, consider starting with a silent brainstorm, to make sure that people’s willingness to talk in a group doesn’t determine their ability to have their opinions heard.

Finally, and this shouldn’t have to be said, but be respectful of differences in religion, diet, sexual orientation, gender identity, family structure, and anything else that’s not directly connected with their performance as your team member. When somebody expresses their preferred pronouns, a dietary preference, or a constraint on their time, recognize that can be a pretty vulnerable thing, especially if they feel like they’re being asked to justify it.

Measure and Improve

While empathy might be tougher to measure than uptime, error rates, or other performance metrics, there are still steps you can take to track how you’re doing.

Our scrum teams each do “health checks” using Google Forms at the end of each sprint, where we smiley-face-rate our feelings about various factors including teamwork and support. Consistent sad-faces tell us there’s something systemic we need to work on. Managers can also specifically ask in 1:1s about how communication is going within and outside the team, to identify patterns.

One additional measure (and benefit) of strong empathy is the amount of diversity you see in your organization. If you can’t connect and communicate with people from different backgrounds, you’re unlikely to be able to recruit or retain diverse talent. On the other hand, when you’re able to see through the beer test, or the hijab, and make potential team members feel included and valued, your pool of awesome candidates gets much bigger.

One final reading recommendation here: Stalling for time: My life as a FBI hostage negotiator. Not only is Noesner’s story a thrilling read, but it takes many of the communication principles above and shows how a master of nonviolent communication permanently changed the FBI’s attitude about the place of empathy in rescuing hostages and defusing violent, high-stakes situations.
July 22, 2015
5 Ways Continuous Integration Reduces Risk in Development

***This post is based on themes and ideas from the great e-book, “Continuous Testing,” by our friends at Parasoft.

There’s a lot of talk about DevOps and Continuous Integration in the tech space right now, but sometimes the implementation of those methods can be tricky.

An effective Continuous Integration strategy requires DevOps teams to manage risks in software development with a range of practices.

Here are five processes you can use to make Continuous Integration work for you.

Automated and Self-Testable Builds

DevOps professionals can use separate machines for software integration to eliminate the tight coupling between the build process and Integrated Development Environment (IDE) that leads to last-minute integration concerns.

Everything needed to build the software should be contained in a separate version control repository.

Dedicated Continuous Integration servers and open source build automation tools can run these builds through continuous testing processes and deploy the product in the required environments.

Automated deployment further eliminates the bottleneck of waiting for other team members to deploy testable versions of the latest software.

Synchronize Version Control Repository

Continuous integration is only possible as long as Dev, Ops, QA and the wider IT team is tightly integrated, collaborative and supportive of each other’s efforts.

From a tooling perspective, effective collaboration requires a separate-but-unified master database updating changes in real-time.

All database artifacts — including scripts, procedures and other assets should be placed in this version control repository.

DevOps teams can enable Continuous Database Integration by using automated database integration, local database sandbox and version control repository to share database assets, as well as empowering developers with capabilities to modify the database and making DBAs a part of the Dev team.

Build a Stronger Foundation and Expand Test Coverage

Limiting test cases to individual assets executed at every check-in to the version control repository ensures the continuously integrated builds are defect-free in combined form.

Tests should be repeatable and performed continuously in virtualized systems replicating unavailable testing environments.

Continuous Integration methodologies allow you to determine how much of the code was actually tested without much manual intervention.

Although some code coverage tools can assess the percentage of test coverage by package or class, Continuous Integration systems ensure this coverage is updated in real-time.

For instance, DevOps teams can run these tools as part of the Continuous Integration server build script every time the version control repository is updated with a code change.

With this information, untested code would never get through to the next stage and the latest test will be conducted across all areas as required.

Enhance Project Visibility

With automated build testing and deployment using a centralized code repository, build feedback mechanisms update Devs and Ops with the latest test results and the necessary progress information.

In cases when the same level of automation for complex builds seems impractical, DevOps teams using manual procedures must establish a feedback loop to bring everyone on the same page with real-time communication and notification of accurate performance and results.

These capabilities ensure the same high standard of reporting is maintained for manual testing practices as available for automated build testing systems.

Follow Consistent Dev Standards

Quality code forms from the ground up, with adherence to common code and architectural standards established for an organization’s specific requirements.

Automated inspection tools enable Devs working on short and quick builds to maintain their focus on quality from the very foundation of the software itself.

DevOps teams should assess every integration build for potential defects that surface after integrating several iterative builds but should be taken care of far earlier during the development stages.

In Conclusion

The right toolset is key to Continuous Integration, as is the cultural change that facilitates this DevOps practice.

Continuous Testing presents several areas of improvement for DevOps teams to develop iterative builds in individual, isolated branches and then merge the latest builds in a unified, single and well-synchronized Master branch.

The idea is to ensure all risks are mitigated beforehand so DevOps professionals are not bogged down with endless needs to rework the product — and that gives them time to focus on true Continuous Integration.

June 9, 2015
Atlantic.Net Credits DevOps Culture to Mental Agility

Atlantic.Net (www.atlantic.net) is a Solid State Drive-based cloud hosting provider with locations in New York City, San Francisco, Dallas, Toronto, Orlando, and (coming soon) London and Singapore, says Marty Puranik, Founder & CEO, Atlantic.Net. The Atlantic.Net cloud system is an intuitive IaaS server environment for customers who are developers. The company backs its 24/7/365 operation with a deep, full stack DevOps integration throughout the organization.

That means that everyone has enough understanding of everyone else’s area within the company and of how everything is orchestrated so that when there is a change for the organization to make, everyone appreciates everything everyone else needs to touch to make the change successful. “That’s versus operating in a silo as only a database administrator or only a web developer,” says Puranik.

The Atlantic.Net software stack that makes change possible includes Linux, Nginx (http://nginx.org/), PHP (http://php.net/) /Python (https://www.python.org/), RESTful APIs for internal apps linkages, jQuery (https://jquery.com/), Ajax (http://www.w3schools.com/ajax/), MongoDB (/www.mongodb.org), node.js (https://nodejs.org), and KVM Virtualization. “We use serial protocols to communicate over Infiniband (a fast type of networking) so I would add SRP or iSER serial communications protocols, as well,” explains Puranik. (Infiniband is typically applied to high-performance computing applications.) Atlantic.Net applies a particular mindset as it uses these tools to serve its software, its customers, and its DevOps culture.

Mental Agility Key to DevOps Culture

Atlantic.Net teams feel that maintaining a mental agility across human resources and encompassing all stakeholders—customers, departments, and vendors, who often run DevOps teams, too—is a principal element of a thriving DevOps culture, according to Puranik. “Mental agility here means the ability to reformulate your best case scenario using new tools or upgrades to existing tools. Mental agility also means not designing systems that are so married to a legacy piece of technology that it is impossible to move them off that technology,” explains Puranik.

Atlantic.Net strives to design software such that they can easily add, change, or remove old and new technologies from the stack without making major revisions to the codebase. To do this, the Atlantic.Net team puts considerable time into mulling over design considerations prior to implementation to ensure flexibility in the relationships between technologies and the development stack. The team takes the additional time in order to abstract different models of how a new feature could work so they can make the best selection for everyone. That’s a better choice than tightly integrating the new technology simply because it’s easier for them to think through and makes development go faster. “In this sense, mental agility means keeping your options open so you don’t end up with your back against the wall and your best path cut off,” says Puranik.

The benefits of this type of mental agility are two-fold: the ability to continually evolve the codebase without any drag or friction from permanent ties to dated development moves; and, the ability to add functionality for customers with minimal code changes and without breaking the code.

Stretching Out Strategy to Satisfy Stakeholders

The nimble minds at Atlantic.Net take stakeholders into account because of the repercussions on customers, employees, and vendors of misguided changes to the software. “If, for example, we alter how a customer must access a particular feature, this creates a learning process for the customer,” says Puranik. If the change doesn’t carry enough value to make it worth the learning process, then the development team should not have put the customer or themselves through the whole mess.

In the same way, adding a new expression of a software component such as an API from a third-party vendor brings the reliability of the vendor into question. “Can the vendor handle the number of queries we are going to make? If not, it’s bad for the vendor, for us, and ultimately for our customers,” says Puranik.

Atlantic.Net must look at the entire development process from everyone’s angle, putting on their shoes and considering the outcomes of development choices. “Sometimes different people inside Atlantic.Net must be the champion on behalf of different stakeholders internally and externally to make the whole team think differently or at least think through what the outcome of specific actions might be,” says Puranik. That way continuous deployment is fast, and harmless—a win-win!

May 29, 2015