devopsy.kinsta.cloud

Tag: development

Release Automation: Bigger than DevOps
Why DevOps is not for everyone and how release automation helps in bimodal world

So am I really going to write a blog calling DevOps a big hoax?

Well, not exactly, but as a follow-up to my previous blog on the difference between DevOps and Continuous Delivery, I want to alert you to a few key facts:
1. DevOps is not always the right methodology for developing and delivering software.
2. Agility and speed can be achieved even in a waterfall method if the right tools are in place.
3. Release automation is bigger than DevOps (yeah, I said it!).
Say No to DevOps?

Okay, so here goes: Why is DevOps not always the right way to go? Simply because not all systems are born equal. For a back-end master customer data system, stability trumps iteration. Fail fast and roll forward simply aren’t sustainable in many of today’s most core business applications such as banking, retail, media, manufacturing or other industry vertical. Clearly, these enterprises still need to innovate and provide new and exciting ways for their customers to engage through web and mobile, but not at the risk of a terminal business failure at the back end.

Delivery capabilities for these back-end systems need speeding so they don’t become the bottleneck for the entire business. However, restructuring teams and changing development along with test and release into tens and hundreds of small/tiny parallel updates cannot be supported by a legacy technology stack (say Siebel or SAP at the back end). It is likely not even advisable at all.

This doesn’t mean we should accept the realities of the last century, resort to the status quo and leave our release cycles to once or twice a year for key applications. Slowly and surely we would lose any competitive edge.

Luckily there is a middle ground, and this is why I believe release automation is bigger then DevOps. That middle ground is release automation. I will explain why, but I need to start by examining DevOps a little more first.

What is DevOps?

At its core, DevOps aims to get new ideas and functionality into the hands of users quickly and iterate fast. It achieves this by fixing two traditional weaknesses in the software delivery life cycle, namely “overhead” and “rework.”

Overhead is anything that developers do that isn’t directly related to developing new features or improving existing ones (manual deployments would be a big item in the overhead column for sure, as would debugging production deployment issues or handling configuration problems).

Rework occurs anytime developers have to go back over a piece of code they already have worked on. However, this does not include bug fixing, so rework occurs when test cycles take a long time and developers already have been working on new functionality, which now needs to be reworked to address a newly found problem.

DevOps tackles these challenges by attempting to shorten the cycles—for example, by adding smaller changes that developers and testers can test in real time or by using tools such as Puppet or Chef for provisioning quickly to test while ensuring consistency across systems.

This relies on Devs owning updates and rollouts across the life cycle, including production, which challenges everything in the way enterprise teams are traditionally organized. It calls for completely new processes, which often are a great fit for newly developed applications and modern architectures, but can be a complete misfit (and regulatory obstacle) for so many other core systems.

I’m sure you can see where I am going with this. Much like the story, “The Emperor’s New Clothes,” DevOps isn’t what people believe it is.

Why Release Automation is Bigger Than DevOps

The goals of DevOps are sound, however, which leads me to my last point: Release automation is bigger than DevOps. Done right, release automation can accommodate diverse organizational needs and different types of applications in this new, bimodal world. It improves the software delivery process and its final product by rooting out manual work, improving cycle times, reducing overhead and rework.

Crucially, release automation enables incumbent businesses to stay competitive and ensure survival without forcing them to completely abandon their DNA across the board in a “big bang” fashion. Such a process could put the very existence of a large enterprise at risk.

Before signing off on this blog entry I want to make one last thing clear: The argument I am raising here isn’t trying to downplay the benefits of DevOps as a valid methodology. Nor am I denouncing any of the DevOps tool chain technologies out there, which are all welcome advances, of course. This year, 2016, is (according to most industry experts) the year in which DevOps adoption goes mainstream, and I fundamentally believe that it is release automation that drives this adoption, not just pure DevOps conversions.

Stay tuned!
April 13, 2016
The DevOps Equation: Agility + Empathy = Quality

I had the chance to catch up with Jeff Sussna (@JeffSussna), founder and principal of Ingineering.IT, ahead of his keynote address on continuous design, scheduled for DevOps Days Atlanta, April 26-27. He discussed the importance of designing for service, responding to the unexpected, and the importance of building empathy across teams.

Derek Weeks (@weekstweets): Today I’m really happy that we have Jeff Sussna joining us for the latest in this series. Jeff, why don’t you introduce yourself?

Sussna: Sure. Thanks for having me. I’m an independent consultant. I’ve been around, since rocks were young or the 80s, whichever is older. My particular background is that I have built systems and led organizations across the entire development, QA and operations spectrum. So, I’m a little unique in that respect.

My current focus is on helping IT organizations and digital businesses really bring together their ops and design thinking so they can not only go faster and not only be more resilient, but provide more customer value at the same time.

Weeks: Let’s talk about that a little. How is continuous design—a subject you have written a lot about—coming into play? How can it help us be more resilient.

Sussna: Just to explain a little about what we mean by continuous design. It’s basically bringing together the entire organization and a unifying design in operations. It’s taking what we are doing in DevOps and expanding it to include product, design and support. It lets the whole organization really have an agile conversation with a market and with its customers.

The issue that we face is that we live in a service world now. The point is no longer delivering products like cars coming off an assembly line. It’s much more about responding to our customers and having relationships with them. This is happening in a world that is more and more complex while less and less is in your control. We can’t figure everything out in advance. We have to have the ability to respond to the unexpected. We have to be able to do that at every level.

“This is happening in a world that is more and more complex, while less and less is in your control.”

I will give you a couple of examples of things that are perfect illustrations of this:

I recently wished a friend of mine happy birthday. The reason I did that was that because there was a notice on my calendar that said it was her birthday. It was wrong. It wasn’t her birthday. I had no idea where that notice came from. Did it come from Skype? Did it come from LinkedIn? Did it come from email? I had no idea.

We face that in IT, too. Systems are becoming more complex. We develop microservices. We use public and private clouds at multiple levels. We’re no longer in a situation where we can control things. We have to be able to respond and respond at all levels.

When we make design and operations unified and continuous, we are able to say, “Where did we think we were going? Where would we need to go? and what do we do next?”

What I talk about in my book, in my workshops, and all my work is that this needs to happen at every level. It’s not just a matter of designing for failure inside the data center or inside the infrastructure. We also have to do it at other levels.

Here is another example:

I was trying to get paid recently by a large enterprise client in Europe. They were trying to wire money internationally and my bank wasn’t really set up for it and they were having problems. They just kind of gave up. I kept getting these messages saying, “Sorry, we tried paying you and it didn’t work.” That was the end of it.

I started to get frustrated and my client within the enterprise started to get frustrated to the point where they sent a scathing email to our AP department which they CC’d me on, saying, “Why can’t we take the lead to get his guy paid?”

That’s an approach that is based on resilience in the sense of assuming that something can go wrong and figuring out what he can do to fix it when it can go wrong. As opposed to the traditional approach where we’ve been trained to push certain buttons and take certain actions. If those actions don’t work the way they are supposed to, well, I guess that is someone else’s problem.

https://www.youtube.com/watch?v=ISSMHmwIMtA

Weeks: This brings to light some recent conversations I was having with Paula Thrasher at CSRA. She was talking about developing systems and how to make them more resilient. She is using pictures as part of her conversations with people to help visual their work. What work steps are there? What work is flowing? What is communicating with what?

Are you taking similar approaches with continuous design in how you work with clients to pursue it?

Sussna: Well, I try to introduce them to the principles and I also try to introduce them to a set of practices which actually comes out of Mark Burgess‘s work on promise theory. It is really a conceptual model for how we think about success and failure and how do we think about service. For example, Amazon helps you get everything you need in your life from shaving cream to batteries to snow tires to books as quickly and easily as possible. In order to keep that promise they make a whole variety of other promises. Some of which filter all the way down to a promise to be able to go over and make a change to production as quickly as possible.

Once you take that mindset. The interesting thing about a promise and the reason we use that word is because it is very intuitive. We all know what a promise is. We all make promises to each other every day. On the other hand, when we think about it in terms of service, IT, or whatever it may seem a little odd. The thing about promises is that on the one hand we are making commitments to do things and on the other hand promises sometimes get broken. That is actually good because it forces us to think. If I promised to pay my vendor. What do i do if I’m not succeeding in paying my vendor?

The thing about promises is that on the one hand we are making commitments to do things and on the other hand promises sometimes get broken. That is actually good because it forces us to think.

It’s sort of flipping our mind. Again, we are doing this in DevOps world where you start thinking about things like mean time to repair (MTTR) instead of mean time between failure (MTBF). We shift from how do we create a situation to how do we create a situation from where nothing will ever break—which becomes less and less possible. When then need to shift to a situation where we can respond more and more quickly.

We go to executives and say, “We don’t want to worry about how often stuff breaks anymore,” and they freak out. We have to teach them and we have to teach ourselves that we can’t prevent things from breaking. Instead, let’s be able to fix them so quickly that maybe no one notices. My focus is on really helping people shift their mindset from thinking about ourselves to thinking about who we are helping. And from thinking about what we are doing to thinking about how well it succeeds and how well it overcomes problems.

We have to teach them and we have to teach ourselves that we can’t prevent things from breaking. Instead let’s be able to fix them so quickly that maybe no one notices.

Weeks: You talked about delivering on promises as quickly as possible. How does continuous design account for doing things as quickly as possible but making sure the quality is there at the same time?

Sussna: In two ways—one way is that when you do things in small increments you are able to find problems more quickly. This is classic; it comes out of lean. It comes out of integration. It’s pretty basic stuff for our community now that it’s much easier to find and fix a bug five minutes after you change a code than five weeks after you change the code.

When you put things like Linux, agile, DevOps, and design all together, they all focus on incremental feedback loops where you discover the gap between what you think you’re accomplishing and what you’re actually accomplishing very quickly. Secondly, when you take this all to the way to the marketplace and you make it part of the relationship with your customer, you can reduce a tremendous amount of waste.

The way to think about continuous design is: it’s all about steering. Think about a boat and you’re trying to sail across a bay. You can’t just point the boat at the other side and say go because the winds are changing and you’re going to be able to respond to things that you can’t predict. If you sail for 20 minutes then discover you had pointed in the wrong direction, you’re going to be way off-course. You’re then going to generate a tremendous amount of energy getting back on course.

What continuous design says is just an extension of continuous integration and continuous delivery taken all the way though the business. When we steer continuously we steer efficiently.

What continuous design is really just an extension of continuous integration and continuous delivery taken all the way though the business. When we steer continuously we steer efficiently.

Weeks: I know you’re going to be keynoting DevOps Days Atlanta coming up in April. Chris Corriere, one of the guys on the organizing committee, tells me that you’re leading a full day workshop on continuous design before that. Maybe you could tell some of the people that are attending DevOps Days Atlanta why they should be there a day earlier for your workshop and what things you’re going to address during that time.

Sussna: To some degree what I am going to address is everything we have been talking about for the last 10 minutes or so—relative to DevOps. I think the key thing is, and we need to think about it. We need to think about more than just going fast. One of the big misunderstandings that I see with DevOps is this notion that it is primarily about delivering application changes to production faster. That’s only half of it. We really need to think about from the perspective of providing service.

One of the big misunderstandings that I see with DevOps is this notion that it is primarily about delivering application changes to production faster. That’s only half of it. We really need to think about from the perspective of providing service.

You know what we are talking about is essentially developers and operations people providing better service to each other. As a result, they can deliver better service to the business.

Where we talk about microservices. You know that word service is in there for a reason. If all you do is you kind of shatter your organization and shatter your architecture into a bunch of little pieces, you still have to figure out how those pieces come back together into something coherent. The answer to that is taking a really deep service-oriented approach. That is very much what my workshop is about. What does that word even mean and how do we do it?

Weeks: You mentioned thinking about interactions as delivering better service to one another within the organization. How does empathy come into your thinking around DevOps and Continuous Design?

Sussna: I’d like to say agility plus empathy equals quality. What I mean by that, in particularly empathy, I think it’s important to take a minute and clarify. When we talk about empathy we are not talking about wallowing in someone else’s pain. We’re not even necessarily talking about things on an emotion or feeling level. It’s very simply the ability to see things as if from an other’s perspective. I will give you a prime example of where that comes into play and how it’s important and how it helps.

Agility + Empathy = Quality

When I work with clients, I tend to look at their process using their organization from the product beginning to the support end. Often times I see support getting the short end of the stick. I see places where they get phone calls from customers with features that they don’t even know exist yet. They sometimes get talked down to by development or product or even operations. They say things like, “Well you’re just supposed to support the stuff. You’re the young people. You’re the less experienced people.”

When we have the ability to see things from their perspective, we can think about how support is actually experiencing our delivery process. If we don’t think about them when we go through the process of delivering new features, they will be lost. Them being lost will show when customers call for help. That will significantly affect customer satisfaction and then even beyond.

The reason we need to empathize with support is so we can empathize with the customer. Often times when we build things and we don’t think about the notion that our customers might have trouble with it. On the one hand we have the illusion that if we design it and we build it really well, they won’t have trouble. We can’t perfectly predict who they are, what they need or what they will encounter. There’s an inevitability to the idea that they are going to need some amount of help.

Weeks: I think that is probably the best description of empathy within a DevOps environment that I have heard anyone talk about. Jeff it has been a great conversation. Thank you very much. I really appreciate it and we look forward to seeing you at DevOps Days Atlanta coming up.

Sussna: Thanks. It has been great talking to you and I am really looking forward to the conference. It will be fun.

If you want to learn more about continuous design from Jeff, be sure to join us at DevOps Days Atlanta. I will also be at the event giving an Ignite Talk, titled, “Real Heroes Draw Pictures,” focusing on lessons learned from the 31 DevOps and continuous delivery reference architectures I assembled.

March 21, 2016
Building a Foundation for Big Room Planning
When people think of agile, the first thing that comes to mind is a software development team. But agile is much more than that now, as every DevOps practitioner knows. Nearly a decade and a half of practice has shown us that high-performing agile software development teams are critical, but not sufficient on their own. When they operate in a silo, software development teams can easily become disconnected from the business and from other teams. To truly achieve the higher business goals, those development teams need alignment, or else they risk delivering the wrong things or the right things but at the wrong time. That is where Big Room Planning comes in.

Big Room Planning is the secret sauce for effective agile planning and execution because it brings together all the people in an organization responsible for delivering software value to market—from development and test to program management and the business—to plan collaboratively. When I say bring them together, I don’t mean that figuratively: The key is to bring them together quite literally, assembling all of the stakeholders together in the same room in real time. That can feel quite shocking because it represents such a significant commitment by the company, but that is the point. Making agile work requires a commitment across every level of a company.

Before the Big Room

So do you just have everyone roll into a large space somewhere in a hotel’s conference area and hit a home run? Clearly, no. Setting up an effective agile delivery group takes time, and is far more complicated than spinning up agile development teams because of all the roles involved and how each team and individual will approach their part of the work. Most importantly, establishing the leadership changes required to truly change the way an organization works must also take place. Bottom line: Establishing an agile mindset throughout the organization is critical to have a foundation for success.

There are three distinct phases of preparation required when standing up an agile delivery group (Agile Release Train, in SAFe parlance), which we call:
- The Ready phase
- The Sync phase
- Rhe Go phase
Each are vitally important, and without success in the earlier phases, your delivery will get bogged down in the latter stages.

Get Ready

The Ready phase is probably the most critical, and it establishes the right leadership, organizational mindset, and right group of people required for agile value delivery. Three key steps include:
- Enterprise value stream mapping
- Agile organization design
- Leadership development
Without these key steps, supported by leadership at every level, you simply cannot instill the cultural and mindset changes required to stick.

Get Synched

In the Sync phase, agile practices are instilled first. Agile practices specific to each role requires training and coaching to set in, and we advocate for a four-to-12-week period to ensure everyone understands their role, the ceremonies they will use to ensure collaboration and what artifacts they will use to plan and execute their part of the overall work.

Once everyone in the delivery group gets it, it’s time to set up a first Big Room Planning event. This typically takes one month of preparation—without which your event is likely to fail.

Preparing for Big Room Planning should focus on the inputs for the event. First among these is the business vision and context—the domain of executives, line of business leaders, etc. This is essentially the “why” behind what the teams will work on. Key initiatives for the delivery group, with clear outcomes defined (often supported by a lightweight business case), must be communicated effectively to inspire the group. It takes time to put this together.

Next, develop a feature backlog that represents what the delivery group will be planning out during Big Room Planning. Doing this well is a more rigorous process than meets the eye. Features are the crucial ingredient because they represent the value being built to both the business and the customer, and they represent the key deliverables teams will produce.

Product managers typically own this step, and if they are not well-prepared, they run the risk of either asking for too much or asking for the wrong things that won’t have the desired impact on business results. At CA, our product managers go through several iterations of backlog preparation involving stakeholders throughout to company to make sure the priorities are clear.

Even with a well-formed and prioritized backlog, effective Big Room Planning requires an understanding of the capacity of the organization to build what’s needed. This requires good estimation and a good understanding of the each team’s velocity. Estimation and a team’s velocity are big parts of what we learn during the Sync phase: Over a few iterations, the amount of work (a.k.a. story points) a team can deliver outcomes and provides an understanding of roughly the right capacity needed for effective planning. Doing this well ensures teams will be asked to plan not only the right features during Big Room Planning, but the right amount of features so that detailed planning can take place with enough time to design stories, resolve dependencies and address risks that emerge.

Then Go

The Go phase actually kicks off with the Big Room Planning event. The event itself occurs over a two-day period, which marks the starting point for agile execution. If formed well, the right people will be in the room to build value fast. With upfront preparation, the group members will understand what they will be building and why it’s important, and will be set up to create a plan that best delivers the value needed.

Planning collaboratively ensures that the plan is well-designed across the group. Most importantly, because everyone has a part in building the plan—with the opportunity to work together across teams and across roles, with the ability to challenge and even veto decisions as they form—everyone ultimately feels a strong sense of ownership and commitment to the final plan. A successful outcome almost always ensues, and the group only gets better from there are they learn to prepare, plan and execute together as a cohesive “team of teams.”

About the Author/Steve Wolfe

Steve Wolfe is Senior Principal, Product Marketing for CA Agile Solutions. He has worked for more than 20 years on software products in both hardware and software companies. His experience spans 10+ years in both software engineering management and product management, including 7+ years on agile teams and delivery groups. Steve’s passions include helping organizations learn how to build the right things fast and reliably by applying agile and DevOps practices at scale. He is a certified SAFe Program Consultant.
March 4, 2016
What If We Had Git for DevOps?

I used to hate Git. Compared to my previous source control systems (SVN, SourceForge and floppies), it was cryptic, opaque and hard to learn. It was comfortable to create a set of files and folder, then snapshot them as a unit. It was simple to understand and improved my individual productivity.

But that snapshot created a huge tangled mess when I tried to collaborate with other developers.

Once I got over the learning curve, Git revolutionized how I wrote code. Small changes made a lot more sense with frequent integrations. I could easily review the impacts of collaborators’ changes. Even more critically, I could keep my work around while I tooled their incremental fixes and improvements. Development—especially open source development—is a team sport and requires tools that facilitate collaboration at the most fundamental level.

But it was not just learning Git; I also had to adapt how I structured projects and collaborated.

In that context, DevOps automation has been missing the equivalent of Git. We need composability!

Can a Git Model Work in DevOps?

We’ve been building great configuration automation tools (Chef, Puppet, Ansible and Saltstack) over the last few years and now there’s a new wave of cloud templates (AWS CloudFormation, OpenStack Heat, Terraform) entering the fray. These tools are like my original source control systems: they create a vertically integrated stack of configuration that works for a specific case and platform. There are ways to merge them into new stacks, but that ends up feeling like creating a whole new stacks.

Why does merging scripts create new stacks? Changes in other components often break each other.

When you reuse someone’s script, their changes can quickly break your carefully built automation. Instead of collaborating, you’re just forking their work. It feels like we’re running around data centers with automation on floppies. When we fix something in one module, it’s very hard to push that back into the community—we have not built clear, functional boundaries between modules that isolate changes in a way that reduces the risk of using other authors’ scripts and makes it easier to contribute improvements.

Isolating changes does not mean we don’t break integration! It just makes it easier to collaborate upstream.

The answer in our experience is to think of ops scripts like Git: Even inside of a larger set, we need to make the parts independent and decomposable. How do we translate Git into ops terms? Repos are like full systems—directories map to components, files to roles. Like Git, when we make incremental changes to any item, the whole system is impacted. However, we can still isolate that change down to a single line so we can manage collaborative work. This works when the impacts of changes are isolated (like a text file) and fails when changes are part of a tightly integrated unit (like an image file).

We already have git with ops scripts. How is this different?

We are suggesting an even more composable approach. Each ops role should be designed as a functional role with clear inputs and minimal side effects. To achieve that isolation, we’ve been building an orchestration platform, Digital Rebar, to feed the right inputs into existing opts scripts. That allows collaboration within a module without unexpected external side effects. It also allows reuse of modules because changes are well isolated.

When changes are contained, incremental and managed, the whole system is more tolerant to multiple authors. In that way, we may build system automation that’s more Git-like.

February 19, 2016
Using DevOps to Avoid Architecting Application Slums
The U.K. high-rise tower blocks and built circa 1950-1980 were heralded as architectural wonders—large concrete towers easily housing the same population as the smaller houses they replaced. With large rooms, excellent views and surrounding open spaces, they initially were popular residences—low cost and futuristic.

But looking futuristically sometimes has a habit of coming back to haunt you.

Due cost-cutting, substandard materials, demanding deadlines and rushed construction practices, these modern-age housing miracles became the new slums of the day. By replicating tower-blocks across the country, town planners unwittingly replicated bad designs everywhere. The results: undesirable housing and elevated crime levels, leading to slums and urban decay. Even the surrounding open areas were neglected because no one had ownership over maintenance and supervision.

In IT, we have our own design and architectural calamities.

There are many similarities between existing IT apps and decrepit old tower blocks. IT monoliths can be hard to scale, requiring costly maintenance to avoid disrepair. They’re also tough to police, with vandalism and theft being a constant problem. Add to this integration issues (IT’s version of unsupervised open spaces and pathways), and we’re left with systems inadequate to meet the needs of modern digital business.

Enterprise architecture must change with the times

DevOps, with its focus on operations and development collaboration across the entire service life cycle, is now seen as the answer to some of these problems—an approach underpinned by agile and lean principles for rapidly delivering high quality digital services. That sounds great, but let’s not forget enterprise architecture (EA), because without it we could still be building IT slums—only more of them and much, much faster.

Of course, many DevOps stalwarts may argue (with some justification) that heavy EA practices and frameworks haven’t evolved to match the pace of agile. Iterative-style development means where you start isn’t necessarily where you finish, necessitating a transition from set-in-stone architectural designs to a fluid mix on continuously evolving decisions. And, if EA can’t adapt, autonomous and self-managed agile teams increasingly are empowered to make architectural calls.

But great developers aren’t necessarily brilliant architects; they’re not hard-wired that way. It means they could be charging down a development path without architectural guiderails, or tactically making team-based decisions that compromise program-level objectives. For example, the scalability and performance needed to support a business strategy of tripling the customer base and increasing satisfaction scores could get lost because development focuses myopically on delivering more functionality.

However, we can’t just lay the blame on developers. Enterprise architects must recognize that any misalignment between their processes (especially the rigid and static ones) and the vision of an increasingly software-driven business will only further isolate the practice.

Architecting new fluid guidelines and principles

The role of EA in an agile context is still up for debate, but this shouldn’t hold us back. By using architects and fluid guidelines means we can maintain the pace of delivery while preventing chaos. The trick of course is to limit architectural over-engineering while providing developers with a minimum set of EA to avoid technical and architectural debt.

Even in minimal mode EA can still help developers identify quickly what should be considered during design. The aim is to guide developers into understanding what architectural facets make up a successful application, and more importantly, how new ways of thinking can support longer and more sustainable software innovation.

Interestingly, these practices can be presented in a “slum avoidance” context:
- Avoid substandard or unsupported materials – The tool chains being used will probably comprise both commercial and open source software, but beware of anything unsupported. Just as the cost of maintaining tower blocks reached unsustainable levels, so too, will support burden on an organization if software isn’t commercially backed, especially in IT operations. Also, work with development to understand where constraints lead to testing compromises—the “construction shortcuts” syndrome, resulting in more defects and technical debt.
- Never use new technologies to rehouse old problems – As tower blocks degraded and vandalism increased, many local authorities tried to contain the situation by housing “problem groups” in the same block. But that just increased problems. Enterprise architects can make the same mistakes—for example, laying down crazy edicts that everything must be containerized, even some legacy apps; not understanding that runtime independence only extends the life of problem applications; or providing no incentive to drive improvements and clear existing legacy IT slums.
- Monitor and manage your sub-contractors – Whatever cloud model you adopt, abstracting away the infrastructure or application stack frees development to focus on coding. That’s great, but don’t forget “PaaS-ing” apps to the cloud doesn’t mean relinquishing ownership of application performance and the end user experience. To this end, architects must act as cloud-brokerage advisors—for example, working to ensure any cloud service provides open APIs, so that monitoring can be seamlessly incorporated into a unified solution.
Of course, architecting great new digital services with DevOps is only part of the answer. The innovations of today may become legacies of the future and many new apps must also integrate with existing systems. It’s essential, therefore, that architecture covers both bases: working in minimal mode to ensure fast construction of quality services, but also applying standards and governance when needs dictate and systems change.

No matter how good your app designs look on paper, bad architecture just means building more problematic systems and never clearing the existing slums.
February 15, 2016
IT DevOps: One Size Does Not Fit All

As absurd as this may seem, some in IT may be longing for the good old days of the mainframe. After all, the hulking machine, with its dumb terminals and thousands of punch cards, gave IT complete control. No citizen developers or business analysts allowed.

Things got more complicated with distributed desktop computing and its different operating systems and requirements. Server-based web apps simplified things once again, but now mobile has made development more complex than ever. Instead of just updating the server, you also now have to update each mobile device—creating apps that work on each and getting changes out to the entire mobile population.

What’s IT to do? Applied properly, DevOps can help.

The Evolution of DevOps

Coming on the heels of agile development—a term first coined in 2001—is DevOps, which was first popularized in Belgium in 2009. Today, 15 years into agile development and six years into DevOps, it is becoming clear that these intertwined terms are quite broad, encompassing attitude and approach, platforms, tools and solutions. Agile development, with its Scrum meetings and quick turnaround cycles, did speed development but made deployment more difficult, because many organizations just couldn’t support fast cycle deployment.

DevOps, with its platforms, tools and practices emphasizing collaboration between IT and development, targets application delivery with continuous testing, ongoing feature development and ongoing maintenance to deliver apps faster. While the intention is well and good, the DevOps process must be task-specific to succeed.

Today’s development environments can be broken down into three categories: low-code, deployment-centric apps and, in a class by itself, mobile. Delivering apps in these three environments involves a broad range of skills and tools. Therefore, the approach and degree to which DevOps is utilized must vary as well—from virtually no DevOps in low-code environments to extended DevOps in mobile.

Low-Code = Zero Ops

In cloud-based, application performance-as-a-service (aPaaS), low-code environments, DevOps can take a back seat. With the vendor managing, maintaining and controlling the deployment environment, the citizen developer or business analyst is empowered to work on GUI development and other features and functions with little involvement from IT or development. In these environments, power users can easily push out low-code apps. A strong DevOps presence here is not necessary because the separate environments typically required to propagate apps does not exist. The business analyst simply makes the changes and pushes them through to production.

Here, IT is best utilized as gatekeepers, getting involved only when needed. We characterize these environments as zero ops—the less DevOps the better. This leaves business users, IT and development free to focus on their strategic areas and core competencies, funneling resources appropriately and enhancing productivity across the board.

Deployment-Centric = Controlled Ops

High-transaction, highly scalable, business-critical global apps fall into the deployment-centric category. The goal of DevOps in these code-intensive environments is to free developers from deployment issues, enabling them to focus on their key competency of writing code. Well-executed DevOps can effectively move apps from development to deployment, enabling tight collaboration between development and IT. In this environment, a well-managed DevOps process can make it easy for the DevOps team to deploy the apps and to manage the deployment moving forward, scaling apps automatically with rules that scale automatically based on usage.

That is why we characterize DevOps in these environments as controlled ops, a collaboration tightly controlled by development and IT.

Mobile = Extended Ops

From development to deployment, maintenance, monitoring and analysis, mobile has introduced new levels of complexity to application delivery. Developers have iOS, Windows and Android to deal with, not to mention multiple versions within those environments. Then there is the development environment itself—web, native or hybrid. Compounding it further are the devices—countless phones and tablets. And deployment options—public app stores, private app stores, file-sharing services. Finally, the end user is engaging the device in ways never before imagined, placing the need to build responsive and compelling user experiences front and center.

This environment calls for extended ops—a highly collaborative environment where development, deployment, maintenance, support and business stakeholders are all integral to an app’s success. In mobile, everyone needs to have a place at the DevOps table, as mobile apps are literally iterative apps—constantly on the move, morphing and changing, driven primarily by end-user feedback. The ability to make seamless updates and user-driven changes at record speed can spell an app’s success or failure. The collaboration, platforms and tools DevOps offers in this environment have never been so critical, as business users now must be involved in the design and development of the app and in analyzing how the app is used, which will dictate changes moving forward. Successful, extended ops welcomes business users, but does not involve them in the DevOps process itself—that’s for IT and development.

In Summary

DevOps can have an impact on the entire application delivery cycle. From improving deployment frequency—which can lead to faster time-to-market—to shortening lead times for new releases with lower failure rates, the goal of DevOps is to maximize the predictability, efficiency, security and maintainability of the simplest mobile app to the most complex transactional system. The key to successful DevOps is applying the right level of collaboration and employing the right tools to ensure that DevOps improves quality and productivity rather than hinders it.

About the Author/Mark Troester

Mark Troester is the Vice President of Solutions Marketing at Progress Software. Mark has extensive experience in bringing application development and data integration products to market. Mark previously led product marketing efforts at Sonatype, SAS and Progress DataDirect. Before moving into marketing, Mark worked as a developer and developer manager for start-ups and enterprises alike.
LinkedIn Google+

February 3, 2016
Boston DevOps Leader Pete Cheslock Talks Accomplishments

Pete Cheslock began his leadership role in DevOps in 2009 at cloud-based email archiving company Sonian. The company was founded by friends and former colleagues of his, who called Cheslock for help when they got funding for the Sonian venture. He filled in wherever they needed him—as did everyone else in the first few years of the 10- to 20-person startup.

“As we grew, I saw a need for leadership for Sonian’s five-person operations team. I went to our VP of Engineering about the need for ownership of operations and said I’d love to be the one to give it a shot,” says Cheslock.

Sonian was an early user of Amazon Web Services, deploying production code with customers by 2008. Sonian had been named an AWS Startup Challenge finalist, but despite its forward-thinking approach to the cloud and infrastructure as code, many of its processes were immature.

“The application had scaling issues and we were spending an inordinate amount of time on reactive measures, dealing with application downtime and using short-term fixes for continuing issues when I took over,” says Cheslock. The young DevOps shop wasn’t able to make headway in innovative, future-looking projects.

Smothering Fires, Forging Ahead

Cheslock transitioned its team into two groups—a proactive group for innovative projects with a view to tomorrow, and a reactive team to get the worst recurring fires under control. Cheslock added himself to the latter group while remaining head of Operations. “While we attacked the hairiest issues, the proactive group worked on tooling that we needed in support of CD (continuous development) and CI (continuous integration), which made deployments much easier,” says Cheslock.

It had been taking the Sonian team about a week to deploy software for all its customer environments. The process of deployment affected much of the software stack; it required testing as well as completing and checking off several items on a lengthy deployment list.

Cheslock directed the proactive team to automate much of that checklist. Soon, they had reduced deployment time to 20 customer environments in just three hours, from five three-hour periods over five days.

Weeding Out Costly Cloud Issues

Cheslock and Sonian next began to work on cost issues. “We built an application to analyze our Amazon usage and correlate it with our Chef server, which was our single source of truth for all instances,” says Cheslock. The app revealed unused and abandoned servers that had been used for development and testing, which were still running and costing the company money.

“We found occasional spot instances where we could pay fractions of the retail price for batch workloads to cover these. We leveraged the Amazon Marketplace to make better purchases of infrastructure to save money,” says Cheslock.

Cheslock was able to shrink 800 instances on a single AWS cloud with a monthly spend of $450,000 down to 400 instances across four public cloud environments with a much smaller price tag of $200,000 a month. He accomplished this in three months, even as Sonian’s cloud operations team doubled.

Now senior director of Operations and Support at Threat Stack, Cheslock is nurturing DevOps methodologies and seeing continued growth and success there.

BosOps

A few years ago, Cheslock and a few friends started a Boston-area technical operations meetup called BosOps, inviting people who managed operations at various companies to get together regularly to discuss their challenges and offer support.

“As BosOps grew, we wanted to reach a broader audience and started pulling in more people in operations or DevOps roles to gather on a quarterly basis,” says Cheslock.

Currently, BosOps has about 150 participants, who also participate in online chats using the free tier of Slack.com, a popular team-working messaging app among DevOps folk. “It’s great for people who are new to Boston,” says Cheslock.

BosOps helps by quickly connecting a professional network for people who are new to the industry or to the area, enabling them to meet like-minded people and to secure a better foothold—perhaps even find a job they like more than their current position.

January 14, 2016
3 Ways Manual Processes are Risking Your Development Cycle

During my two decades in IT, I’ve seen the demand for Agile and its associated methodologies grow. At the same time, however, I’ve witnessed the database being left behind.

DBmaestro, where I am co-founder and CTO, recently conducted a survey titled, “Database Deployment & Development Risks,” to gain insights into the risks associated with various database development practices. The survey focused on manual processes used throughout the development cycle.

The results revealed two divergent realities: Despite respondents, from diverse sectors, overwhelmingly recognizing the risks associated with manual process for the database, many organizations continue to use processes and solutions that don’t fully automate the development cycle.

One area where this was particularly apparent was with the source control. It’s imperative that development teams are able to trust their source control. But even when standard source control solutions are being utilized, 70 percent of those surveyed are still interjecting manual processes.

What that means is that even though many of these professionals have some type of solution, they still see a clear and present risk because:

1. When integrating changes, it’s possible for one developer to wrongly override valid changes with older revisions from other development branches, a potential concern that nearly 90 percent of survey respondents associate with risk. Still, more than half (53 percent) of the IT professionals surveyed integrate changes, creating the possibility that such overrides may occur.

Nearly three-fourths (70 percent) reported having a database source control, but 53 percent reported that they continue to have problems that a more adequate source control solution should resolve. In this case, there is a problem with the source control tool implemented.

2. The database may not be in sync with the version control repository. Ninety percent of the IT professionals surveyed associated the potential for these sources to be out of sync with risk. What’s more, 72 percent of those surveyed admit that their database may not be in full sync with the version control repository.

3. Introducing changes without first verifying each database object vs. its version control revision to ensure each database object is up to date is a process that nearly 90 percent of survey participants associate with risk. Despite a widespread awareness that failing to verify accuracy between database objects and database source control before introducing changes is a risky practice, 53 percent continue to do so.

That means that the remaining 47 percent of respondents do not trust their source control solution regarding the database code and are implementing steps to validate it matches the database before introducing changes.

While interjecting manual steps can help alleviate the risk associated with the possibility of overriding critical changes in the database, it’s time-consuming, and manual interruptions to the continuous delivery cycle introduce risks of their own. When you can’t place full trust in source control, two developers working on the same database code or objects can easily override or revert critical changes introduced by the other developer when the script is updated in the database—sometimes with disastrous consequences.

Only a Database Enforced Source Control solution that covers database lifecycle management (DLM) from development through build and deployment can create a “Single Source of Truth” for your database development assets. This allows you to gain trust in your source control and implement the rapid changes today’s development cycles demand—without plugging the process with time-consuming manual steps. Database Enforced Source Control promotes complete trust in your source code, allowing developers to work simultaneously on the same database objects without the risk of overriding important changes when scripts are updated, promoting true synchronization across dispersed development teams.

Download the complete results of the 2015 “Database Deployment & Development Risks.”

About the author:
Yaniv Yehuda is the Co-Founder and CTO of DBmaestro, an Enterprise Software Development Company focusing on database development and deployment technologies. Yaniv is also the Co-Founder and the head of development for Extreme Technology, an IT service provider for the Israeli market. He was a captain in Mamram, the Israel Defense Forces computer centers, where he served as a software engineering manager.

January 6, 2016
How to lower risk in agile development
The increasing complexity of application development and subsequent rise of Agile development is changing the risk appetite in many software development organizations. Unless you’ve got an application which is highly regulated or mission-critical, it’s rarely feasible or sensible to manage Quality Assurance within the zero-bug environment often associated to Waterfall. Software development teams can become handcuffed by the lack of flexibility and struggle to maintain a focus on delivering business value. Agile methodologies paired with DevOps emphasize speed, rapid iteration and a “fix as you go” mentality over driving toward a perfect product out of the gates. So how can we lower risk in agile development?

There are other contributing factors to the growth in software risk, notably the continuing adoption of cloud computing. Given the appealing economics and flexibility for updating and repairing code in the cloud, many teams will choose to host even the most high-risk data on the cloud. Conversely, teams must focus even more attention on security, as the cloud exposes more data outside of the company’s network.

Software interdependencies are another risk; many business processes today rely upon the use of outside services from APIs to OAuth authentication. Teams must be on top of changes in these external systems to ensure that critical functionality can still operate. This is especially true in a mobile context, where even the operating system and hardware components are changing at a rapid pace.

There doesn’t have to be a trade-off between speed and quality. It takes a change in mindset and processes, however: managing software risk in the age of Agile and cloud is a whole new game. Here are a few things to consider:
1. Balance risk with business needs: A shining reason behind Agile’s popularity is the unceasing demand for speed and business alignment. Software and IT organizations have realized that in order for their employers to be competitive, they must operate in ways that are highly business-responsive and adaptive. Agile also helps organizations address the complexity of software development today, given the impact of mobile devices, the proliferation of new languages and tools for Web apps, growth in cloud-based computing and the overall importance of digital processes to a company’s success. QA teams need a more educated and informed approach to testing. That means taking on a measured approach to risk, wherein risk management tasks are implemented with business priorities in mind rather than as a one-size-fits-all strategy.
1. Risk-based testing: In order to take on more risk as part of Agile and DevOps thinking, there must be proper checks and balances integrated within the testing process. Instead of testing every feature and function, and shifting timelines, prioritize testing and quality management measures to fit business priorities. Risk-based testing commonly assigns a risk rating score for individual tests, which gives developers and QA an idea of the level of risk involved in delivering the code covered by that test if the test is skipped. As a result, the product team might spend more time testing versions of the application for devices which comprise the most important or largest sector of the customer base. However, common device types and operating systems are not the only variables that can factor into risk scores. For example, if one particularly large customer relies on a key feature or function, that should mandate additional testing around that area of the application. By linking business drivers to development and testing, organizations can still deliver quickly yet with the assurance of superior results in the areas that matter most to revenues and customer satisfaction. Ongoing testing of high priority application features also results in the avoidance of costly, wholesale changes down the road.
1. Integrate development and testing: As mentioned earlier, integration between development and testing is critical for many projects, yet it is frequently overlooked when managing Agile teams. To maximize success, there needs to be a deeper understanding between coders and testers on potential ripple effects of introducing new features as they are planned. This can occur during the everyday processes of Agile collaboration and sprint planning. Testers should know early on which are the top risk areas affected by a piece of new development; testers and developers both should incorporate development best practices which focus on understanding and planning for interdependencies, such as designing code that limits the downstream impact of integrations.
1. Best of Breed tools: The market for tools has expanded vastly along with the onset of cloud, mobile, social and big data technologies and the DevOps and Agile movements. The average IT shop is using dozens of tools for application lifecycle management (ALM), build, automation, project management, change management and testing. The newer tools, designed for Agile and Web development, allow for rapid iteration yet still provide controls and collaboration features to minimize risk. Beyond functionality, look for tools that will integrate with your existing toolsets. This will allow for faster, real-time visibility on application deliverables, new features and any quality issues that have been discovered.
Software development is more complicated today than ever. The good news is, it’s much less expensive, faster and easier to make a change in the code compared with even a few years ago. Incorporating some risk into a product is good, because it means that innovation and time-to-market are top of mind. However, introducing too much risk in high-priority customer areas or critical periods in a company’s growth can sink a business. Understanding the difference is what can allow you to keep both your customers and executives happy, with a brisk yet manageable pace of innovation.

About the Author/Kevin Dunne

Kevin Dunne is the Director of Product Strategy at QASymphony, ensuring their continued commitment to innovation and delivering tools to create better software. With a deep interest in the emerging trends in software development and testing, Kevin is dedicated to collaborating with thought leaders in this space.

Kevin comes to QASymphony from Deloitte, where he managed testing on large government and Fortune 500 engagements delivering ERP implementations and custom software development. As one of the first employees at QASymphony, Kevin has seen many facets of the business working in sales, customer support, marketing, and product management.

Kevin holds a Bachelor of Science degree from Vanderbilt University. Connect with Kevin on twitter.
August 20, 2015
5 Ways Continuous Integration Reduces Risk in Development

***This post is based on themes and ideas from the great e-book, “Continuous Testing,” by our friends at Parasoft.

There’s a lot of talk about DevOps and Continuous Integration in the tech space right now, but sometimes the implementation of those methods can be tricky.

An effective Continuous Integration strategy requires DevOps teams to manage risks in software development with a range of practices.

Here are five processes you can use to make Continuous Integration work for you.

Automated and Self-Testable Builds

DevOps professionals can use separate machines for software integration to eliminate the tight coupling between the build process and Integrated Development Environment (IDE) that leads to last-minute integration concerns.

Everything needed to build the software should be contained in a separate version control repository.

Dedicated Continuous Integration servers and open source build automation tools can run these builds through continuous testing processes and deploy the product in the required environments.

Automated deployment further eliminates the bottleneck of waiting for other team members to deploy testable versions of the latest software.

Synchronize Version Control Repository

Continuous integration is only possible as long as Dev, Ops, QA and the wider IT team is tightly integrated, collaborative and supportive of each other’s efforts.

From a tooling perspective, effective collaboration requires a separate-but-unified master database updating changes in real-time.

All database artifacts — including scripts, procedures and other assets should be placed in this version control repository.

DevOps teams can enable Continuous Database Integration by using automated database integration, local database sandbox and version control repository to share database assets, as well as empowering developers with capabilities to modify the database and making DBAs a part of the Dev team.

Build a Stronger Foundation and Expand Test Coverage

Limiting test cases to individual assets executed at every check-in to the version control repository ensures the continuously integrated builds are defect-free in combined form.

Tests should be repeatable and performed continuously in virtualized systems replicating unavailable testing environments.

Continuous Integration methodologies allow you to determine how much of the code was actually tested without much manual intervention.

Although some code coverage tools can assess the percentage of test coverage by package or class, Continuous Integration systems ensure this coverage is updated in real-time.

For instance, DevOps teams can run these tools as part of the Continuous Integration server build script every time the version control repository is updated with a code change.

With this information, untested code would never get through to the next stage and the latest test will be conducted across all areas as required.

Enhance Project Visibility

With automated build testing and deployment using a centralized code repository, build feedback mechanisms update Devs and Ops with the latest test results and the necessary progress information.

In cases when the same level of automation for complex builds seems impractical, DevOps teams using manual procedures must establish a feedback loop to bring everyone on the same page with real-time communication and notification of accurate performance and results.

These capabilities ensure the same high standard of reporting is maintained for manual testing practices as available for automated build testing systems.

Follow Consistent Dev Standards

Quality code forms from the ground up, with adherence to common code and architectural standards established for an organization’s specific requirements.

Automated inspection tools enable Devs working on short and quick builds to maintain their focus on quality from the very foundation of the software itself.

DevOps teams should assess every integration build for potential defects that surface after integrating several iterative builds but should be taken care of far earlier during the development stages.

In Conclusion

The right toolset is key to Continuous Integration, as is the cultural change that facilitates this DevOps practice.

Continuous Testing presents several areas of improvement for DevOps teams to develop iterative builds in individual, isolated branches and then merge the latest builds in a unified, single and well-synchronized Master branch.

The idea is to ensure all risks are mitigated beforehand so DevOps professionals are not bogged down with endless needs to rework the product — and that gives them time to focus on true Continuous Integration.

June 9, 2015