devopsy.kinsta.cloud

Author: Jonathan Fletcher

“Culture” isn’t a swear word
I’ve heard a number of lazy criticisms of DevOps recently particularly where individuals talk about culture and that gets my goat.
“It’s fluffy”
“It’s in-actionable”
“DevOps is just automation”
“Cultural change isn’t relevant in a bank – its a term used by the Facebook’s of this world”

I think part of this is the continuing confusion over what the term means, along with DevOps being used as an umbrella term for a dozen different other things (Continuous Delivery being the most prevalent).

I’ve talked at length about what DevOps is to me in my blog so I’m not going to cover that again. However, I am going to talk about this wholly word “culture”.

Let me start with a great definition of what culture is (it’s not my quote but I wish it was) – apologies I forget the source.

“Culture is what you do when there is no one around to tell you what to do”

I think that’s brilliantly simple and bang on the money.

DevOps is concerned with bringing two disparate groups together (unsurprisingly Developers and Operations) and encouraging empathy and collaboration between them.

Nothing in that talks about automation.

The companies with the right culture between Dev and Ops might land as automation as a solution to curing some of the challenges of unshared incentives and goals. However, it doesn’t predicate it.

A strong DevOps culture might turn to Continuous Delivery as a process solution to meeting their demands for an increased pace of change. But again it doesn’t predicate it.

Its really important that people understand the differences and goals between DevOps, Agile, Continuous Delivery etc.

They all very closely relate and link but they are different things (take a look here for a glossary).

Back to those criticisms…

“It’s fluffy” – Culture is closely linked to human behavioural patterns and humans are very different. So its a fairly loose term but does it make less important? Not for me. How your employees and colleagues behave and interact is THE most important thing in a company. There are a squillion clichés out there about how your employees are the most valuable asset but its absolutely true. Great people and accomplish great things. Great technology is nothing without great people.

“It’s in-actionable” – again this is a very lazy criticism. Culture is actionable its just hard to change. Changing the way you recruit, sitting people together in cross-functional teams, not micro-managing, encouraging lunch & learn presentations etc are all actionable ways of affecting culture. Culture is hard to change though as we are dealing with people. People have different reactions to change and are sometimes difficult to second guess. It takes time and effort but that doesn’t make it in-actionable.

“DevOps is just automation” – think I’ve done this to death but for clarity…the right culture could well lead towards automation as a solution.

“Cultural change isn’t relevant in a bank – its a term used by the Facebook’s of this world” – why is having high performing people in a high performing environment only relevant to the unicorns of this world (i.e. Netflix, Facebook etc)? Its a daft comment in my eyes.

Besides who are the companies earning the big bucks these days? Oh yeah its not the banks it’s Facebook, Google etc…

Characteristics of a DevOps culture

Here are my top twelve characteristics of a DevOps culture. How far are you off?
- Measure everything, always
- Individuals have empathy for the rest of the team (i.e. they don’t pass the buck)
- Shared goals and incentives
- Don’t reward the fire fighter, reward the fire preventer
- Reward innovation and challenging the status quo
- Don’t punish people when they try something new but fail
- There is no IT and “business”. IT as much “the business” as the sales people.
- Seek to break down silo’s
- “It’s not my job” doesn’t exist
- “Its my server/code/network/database” doesn’t exist
- Individuals are empowered to make decisions
- Top-up management rather than top-down
September 24, 2015
The 8 “why’s” of Cloud
Myself and the gang have been talking about Cloud a lot recently as we are embarking on a multi-year journey. Cloud is fashionable but is this another cool toy for the techies? As an architect I think the most powerful question in my toolbox is “why”.
- Why are we doing this?
- Does it add business value?
- What are we trying to achieve?
As IT folk it’s instinctive to go straight into the “how” but its the “why” that is the really important question. Get that right and how to do it should be relatively easy. A solution is much easier to put in place if you’ve really captured and understood what it is you are trying to accomplish. I think there are 8 reasons why companies commonly move the Cloud – I’m specifically thinking about IaaS and PaaS here but I guess this also could apply to SaaS in parts.

Scale

The massive amounts of processing power or data storage that public Cloud offerings can achieve would be difficult to achieve building your own solutions (certainly at any kind of realistic price point).

Elasticity

The ability to expand and contract resources (such as compute) depending on your requirements. Traditional IT departments buy capacity in advance which is unused until it is required. The elasticity of Cloud allows us to buy what we want, when we need it and not be constrained by fixed overheads.

Cost saving/model

Some companies believe that moving to the Cloud will save them money.

I think the ROI for Cloud is really difficult to ascertain though. How do you compare apples for apples when Cloud is a different model to on-premise?

It might be easy to say a server costs X to run each year (power, data centre costs, depreciation, support etc) and compare that to a Cloud server. However, what is the cost of having to wait for 3 weeks to get a new server on-premise when you can get it in 5 minutes in the Cloud?

It’s not an easy calculation to be made and I don’t think I’ve seen a really convincing argument from either side.

One insurance company I know said they did some calculations and Cloud was 3x more expensive than on-premise. I’ve seen Cloud companies boast that IaaS is 3x cheaper than on-premise. You should do some due diligence on the cost of Cloud of course. However, in my eyes it’s too complicated and fuzzy to create a compelling story at the moment. Does this mean you shouldn’t move to the Cloud? Absolutely not – but focus on the other benefits too as cost alone is too difficult to accurately determine. The move from CapEx to OpEx is also a difficult question for many and really needs to be thought about. How do you accurately budget for a service if you are using the elasticity and scale features of the Cloud for example?

Future proofing

The rate of new features hitting Cloud providers is pretty astounding. AWS for example have over 450 services now and more hit the shelves every month. You probably won’t need the majority of these features now but what about in the future? As an IT department we can’t predict the future but we can help to respond to it. Why build it yourself when you can buy it as a commodity?

Time to market

With Cloud being pre-packaged and largely automated the time to market for adopting resources is often significantly lower than doing it yourself. You might have a really neat automated server build at your company but chances are you still need to wait for the storage guy or the network guy to do something. Often its not even the time waiting for something to be spun up – its the time waiting for a person to become available in order to press the button to spin it up. For example, in my company it takes 2 hours to build a VM. However I might not be able to book a resource to kick the process off for 2-3 weeks depending on their availability. I don’t think this is a problem that only public Cloud can remedy but again why build it internally?

Resilience

This is a bit contentious but bear with me. Cloud outages happen. On-premise outages happen. However, I would argue that due to the scale of some of things like Azure (and the ease of employing it) that cloud is more resilient than traditionally hosted IT. But…. Only if your application is written/configured in such a way to take advantage of the multiple availability zones and the resilient functionality that Cloud providers…..errrm….provide. If you simply lift and shift from on-premise to Cloud you are just moving the problem. If you treat Cloud just like another data centre you’ll get just another data centre with much the same problems as you currently have. Perhaps even worse. At least in your data centre if there is a problem you are in charge of your infrastructure guys and can prioritise (shout at) them. What is going to happen when you are speck on the Azure infrastructure? Is your voice going to get heard? Is it heck.

Increasing competitive advantage

Although there is value getting your IT teams to run infrastructure (it’s likely your business couldn’t operate without it), does they offer competitive advantage? What competitive advantage do you realistically gain over your rivals by running your own SAN, servers etc? Wouldn’t it be better if your teams were concentrating on high-value tasks that separate you from competitors rather than undifferentiated heavy lifting? Competitive advantage can be realised through such things as…
1. Lowering costs
2. Improving quality
3. Increasing the pace of change
4. Improving end user support
5. Innovation
6. Spending more time understanding the business
Wouldn’t you want your teams spending time on the above rather than worrying about provisioning blades and making sure backup tapes get revolved?

Geographic reach

As companies grow and their worldwide reach increases (customers and internally) then it means getting data closer to the end user.
- Your Google ranking is probably going to suffer in the US if you are serving your website from the UK.
- Your Citrix session is likely to be “sub-optimal” if you are sat in London trying to access your VDI farm in Australia.
- How do you organise downtime for maintenance when your worldwide presence is open 24×7 because of your global reach?
The global reach of public Cloud companies enables to provision resources nearer the people that need them.

A word of warning

In order to fully realise the benefits of the Cloud (such as scale, resilience) applications need to be architected accordingly. You are unlikely to realise the benefits I’ve talked about simply through “lift and shift”. It’s also worth mentioning that people, process, technology, culture all need to be addressed as part of the a Cloud adoption. This isn’t just a technical exercise and nor is Cloud a panacea for all of the challenges that face IT.
April 10, 2015
DevOps – what does “good” look like?
I’ve blogged a lot about my definition of DevOps and what it means to me.

I’ve always thought it much larger than “Dev” and “Ops” and that there isn’t really an end state – it’s an ever changing idea that will adapt to meet the needs of the business.

That said, I’m asked quite often what “good” looks like? How do you know you are on the right path?

For me DevOps is a primarily a cultural idea. If the culture is right then the technology and people will follow. I talk to a lot of people who see this word “DevOps” and reach out straight to technology. DevOps isn’t another word for automating Cloud servers…

In order to an effective “DevOps” organisation I think you need 4 main pillars:
1. Culture – does the culture of your company promote collaboration, self improvement etc?
2. Process – you need to have processes that support different silo’s working together to achieve a fast pace of change
3. People – if you don’t have the right people then it doesn’t matter how great your technology is. People and culture are very linked but I’ve tried to separate them here to demonstrate qualities that individuals hold vs values that the company holds
4. Technology – what are the building blocks you need to be a mature DevOps enterprise?
The following is very much my view of what “utopia” looks like. The truth of the matter is that DevOps means different things to different people depending on what you are trying to achieve. For example, you absolutely don’t need to use public Cloud to achieve DevOps. However, I do think it’s a good pattern to promote working on high value tasks (rather than undifferentiated heavy lifting**) and using a common change process between development and Ops teams.

This is where I want my company to get to…

Culture
- Measure everything, always
- Individuals have empathy for the rest of the team (i.e. they don’t pass the buck)
- Shared goals and incentives
- Don’t reward the fire fighter, reward the fire preventer
- Reward innovation and challenging the status quo
- Don’t punish people when they try something new but fail
- There is no IT and “business”. IT as much “the business” as the sales people.
- Seeking to break down silo’s
- “It’s not my job” doesn’t exist
- “Its my server/code/network/database” doesn’t exist
- Individuals are empowered to make decisions
- Top-up management rather than top-down
People
- Restlessnes (relentlessly looking to improve themselves, others, processes etc)
- Good technical ability across a broad skill set (understand as much of other people’s jobs as possible)
- Everyone can code and use version control
- Everyone understands the test triangle
- Dev-test pairing
- Organised in small product focused teams rather than technology silo’s (align teams to the business not the technology)
- Common incentive schemes
- Favour automation and repeatability above anything else
- CIO/CTO are DevOps biggest guardians & SME’s and seek to destroy anything that affects that culture
- Natural face-to-face influencers rather than endless emailers
- Natural sharers of information
- Take an interest in their specialism outside of work (i.e. go to conferences and take part in the wider community)
Process
- Agile
- Continuous Integration
- Continuous Delivery
- Lean
- Fail-early, fail often
- Release management team are facilitators of change not guardians of change (i.e. they try and aid change rather than stopping/slowing it)
- All change (I mean all) goes through the pipeline from left to right (dev, test, acceptance, production)
- Knowledge sharing and “just enough” documentation is part of the process
- Measuring success and failure is part of the process
- Retrospectives are part of the process
Technology
- TDD/BDD everything (including Puppet etc)
- Everything is in version control (code, automated tests, server config etc)
- Release automation tooling
- Convergent desired state tooling
- Cloud (private or public but the ability to spin servers up on demand)
- The same trending, monitoring & alerting solutions available through nonprod & prod
- Application Performance Management
- Service Virtualisation
- Continuous Integration
- Continuous Unit tests
- Continuous Service level tests
- Continuous GUI tests
- Performance testing
If you can get half way to delivering all of that then I think you should be in a very good place indeed.

It’s a long hard journey if you are a traditional Enterprise. Nothing worth achieving was ever easy though!

I’m outta here for what proved a fantastically successful 2014 (personally and work). 2015 is a massive year too for me but good luck in whatever you have planned ahead.

Happy New Year to you all!

J
xx
January 6, 2015
Daddy DevOps
I found out this week I’m going to become a Dad.

It’s a strange mix of emotions from excited, to terrified, to panicked and happiness.

Not too dissimilar to undertaking a big new project, scoping out how complex it is, getting close to go-live and then delivering it 🙂

[if any one you say I should use Agile processes for childbirth go to the back of the class – having a baby is definitely Waterfall]

At the moment I have time on my hands, which is probably one of the reasons I blog. Time also gives you the room to think and ponder. And because I spend all day, every day talking about DevOps I tend to think about it even in my downtime (and sleep) which is fairly plentiful despite a busy day job.

With my partner away this weekend, bored of playing on the Xbox and all my friends busy/married/parents, I’ve done a fair bit of sitting around of late.

I got around to thinking about how DevOps might compare to parenthood and I think there are some good analogies there.

My goals of being a good Dad are:
1. Ensure that the family is provided for
2. Ensure the health of my child
3. Ensure that the baby is happy
4. Ensure my partner is happy and supported
5. Ensure that the family are safe
My partners goals are probably very similar so we shouldn’t have to worry about pulling in different directions too much.

So that’s cool, we have 2 different parties (Mum and Dad) who’s goals and incentives are aligned and will work together to accomplish them. The missus might not know it, but she’s soon to be living DevOps!

I want to be a good Dad (and partner) and I’ve been thinking about what I can do as an individual to contribute towards our shared common goals? And how do I make it easier for my partner to achieve her goals?

Commonly, DevOps teams turn to automation in order to break down the silos between them and meet their common goals.

If I could automate the changing of a nappy that would help!

I’m not sure I can automate nappy changes (I’d be very rich if I could). Does anyone know if Puppet have a Pampers module?
```
    
    nappy { '/baby/bottom/':
      ensure => clean,
      owner  => 'parents',
      type => 'huggies',
      source => 'crib',
    }
```
Alright so I’m going a little off topic here. My point being that we have two different resources who goals are the same (i.e. ensure the health and happiness of little Johnny).

The stereotyped Dad in the 1920’s that went to work and had little to do with bringing up the child is a good example of a silo.

I’ve gone to work, it’s your responsibility to look after the child – it’s not my problem

Apart from the fact I want to be involved and I want to help my other half, I’m incentivised to help as much as possible because if I contribute then ultimately the happier we’ll all be.

That’s DevOps. Changing your behaviour in order to meet shared goals.

Super excited. Lets just hope the delivery isn’t Continuous.

As ever, please feel free to take a look at my blog http://enterprisedevops.blogspot.co.uk/ or feel free to get in touch if you have any comments.
August 19, 2014
Moving to DevOps in a traditional enterprise
Introduction

One of the challenges of introducing DevOps in the setting of a traditional (i.e. siloed, waterfall etc) Enterprise is where to begin.

It’s certainly something I wrestled with alongside the CTO for quite some time.

It’s not possible to impart years of experience of moving a large company to a DevOps ways of thinking but I hope to give the main 3 pillars of how I’ve approach it previously.

Note…none of it mentions technology and tools! DevOps is primarily a cultural idea and therefore affects people above anything else.

Step 1 – bring in the Rock Star

If you are moving to DevOps in your organisation, then bringing together different people from different silos and then expecting them (under their own steam) to magically cooperate and agree on everything is a little unrealistic.

If you are dealing with historical teams and employees – which I suspect most of you are – then breaking down silos is likely to be a significant challenge.

Aligning incentives makes things a lot easier. If both your devs and ops guys are paid a bonus depending on how much “change” they do in a year is a simple way of getting them to agree on whether they should automate releases for example. That’s a little too simplistic but hopefully you get what I mean.

This may well come at the cost of something else though. The first thing that springs to mind in the last example is quality.

Let’s put everything straight into live! We get paid for it!

Ok so that isn’t going to work.

Sometimes, at least to begin with, you need to bring people together and ultimately empower someone to make decisions when one can’t be reached through consensus. I’ve seen a real danger in DevOps of death by committee. The more silos you bring together then potentially the greater the possibility of them not making a decision.

You may have heard about this idea of DevOps rockstars. Guys (and gals) that get DevOps inside and out. People that understand all sides of the coin (development, DBA, infrastructure etc).

They try and influence through education and persuasion and point people in the right direction. They help the silos reach the right decisions themselves rather than a top-down “do what I say”.

All said and done sometimes this open inclusiveness doesn’t always work and someone needs to just stand up and be counted.

So what’s my point?

If you are trying to move towards DevOps then hire a rockstar (I doubt Rod Stewart is suitable btw). Someone that has done it before. Someone with a very mature sense of what DevOps is and preferably someone that has done most of the jobs he/she is trying to influence.

Moreover, hire someone who’s first instinct is to try and bring people around to their way of thinking. And lastly, someone who isn’t ultimately afraid to put their privates on the line and make a goddam decision.

Step 2 – look at your people structure

The traditional definition of DevOps is to break down the silos between Development and Operations so that they accomplish shared goals.

However, I’m a keen advocate that DevOps isn’t just Dev and Ops (see http://enterprisedevops.blogspot.co.uk/2014/07/devbiztestreleasethingyops.html).

It’s about all the functions that come together to achieve meaningful change.

The devs, the ops guys, the business and techies should all have shared goals and incentives. Where possible they should be sat around the same desk. The team should rise and fall collectively (there is no “us and them”). The team should manage change from cradle to grave (requirements, build, test, deploy, support).

I think we should rename Devops to “DevBizTestReleaseThingyOps”. Catchy.

To that end, DevOps is as much to do with “people” than it is about new tools and processes.

With that in mind, potentially you are talking about some fairly radical changes across a wide number of people of various skills, experience and personality types. I guess it really depends on what your organisation is structured like, the skills of your team and many other factors. For my company (a large UK insurance company) however, it’s meant some fundamental changes.

We started by aligning change functions (devs, testers, BA’s etc) to individual business units (rather than monolithic shared group services).

That fundamental change has been sponsored on the back of some early and small wins by implementing things like Continuous Integration and labeling it “DevOps”. Ok so it’s not DevOps but if using that term starts to get visibility and movement within your organisation then I don’t think that’s a bad thing. I always believed we needed a banner for the modernization of the department to fly behind. If that banner reads “DevOps” then so be it.

That early small success was really vital to us.

When we started out we had a lot of push back and negativity….
- Why are you telling me how to do my job?
- You are using the wrong tools
- I don’t see the benefit
- I know more about this than you do
- What you are trying to do is easy
- We don’t have the right people
- This is a hipster idea for the likes of Facebook
- There are regulatory reasons this won’t work
Basically everything under the sun.

That’s stressful when you are trying to do the right thing and move the incumbent forward.

I don’t think we helped ourselves too much to begin with. There was too much of my team trying to pick up and change other team’s technologies and processes. Not enough leading the other team to water and helping them implement something themselves. Lesson learnt.

Step 3 – create a team to help others

I mentioned earlier that DevOps is cultural idea and not a job title. Definitely not. Unimaginable.

So how is it I run a “DevOps” team? Surely that’s a collective of “DevOps engineers”?!

My team “Platform Services” supports the idea of the change platform at my company. The change platform consists of tools such as Jenkins, Puppet, uDeploy, Teamcity, TFS and a few others.

Someone ultimately needs to “own” the tools making sure they are maintained, creating best practices and establishing a roadmap for the future.

My team goes beyond this and helps different change functions adapt to the change platform and are evangelists for things like DevOps and Continuous Delivery.

With DevOps being concerned with breaking down silos its contradictory to have a team that bottleneck of knowledge and skills. To this end, Platform Services is the silo-buster – we help others pick up the tools & processes and implement them.

For example, we don’t do deployments ourselves but we do sell the benefits to other teams and help them to adopt the tools and processes.

I think this is a good model and helps spread the success of DevOps around the IT function.

It helps that consultancy from my guys is effectively free to other parts of the IT team (thanks to an enlightened CIO & CTO).

I hope you found some of that useful. Feel free to check out my blog on my move to DevOps: http://enterprisedevops.blogspot.co.uk/

Good luck and get in contact if you have any questions or comments.
August 4, 2014