devopsy.kinsta.cloud

Tag: collaboration

Digital Disrupters – Building a DevOps Leadership Profile
While the advancement of DevOps methodology remains fairly nascent for many organizations, an ever-expanding class of leading practitioners are evolving a wide range of related best practices.

Beyond the well-worn concept of “unicorns” – the scant few organizations that stand as mythical examples of DevOps nirvana – this broader set of early adopters have begun to pull back the curtain on their efforts, lending further detail to the strategies that have bred this current model for success.

Last week, industry analysts Freeform Dynamics published valuable research that highlights the ongoing work of such DevOps innovators, dubbed as “Digital Disrupters” for their accomplishments in pursuing larger digital transformation. The report “Exploiting the Software Advantage” – sponsored by CA Technologies, outlines a critical set of initiatives crucial to navigating the DevOps landscape.

What are the specific hallmarks of those organizations leading the charge? Where are others struggling to catch-up? Many of the Freeform Dynamics survey results, based on 1442 respondents, reaffirm widespread DevOps observations, while other findings serve to shed new light, including:
- Everyone builds software: Perception is, in fact, reality and to deeply engrain digital transformation, Digital Disruptors are increasingly identifying with this mantra. While 95% of leaders have committed to becoming more app-centric, only 51% of mainstream organizations have adopted this persona. Similarly, 84% of Disruptors directly identify as “software companies”, regardless of industry, compared to 40% among the masses.
- Transformation is required: This point may seem obvious, but it certainly bears repeating. A whopping 92% of leaders have actively moved to break down traditional silos between Dev and Ops, with 51% of those organizations having already completed this work. By comparison, only 18% of mainstream organizations claim to have done the same, supported by 59% who at least share this more collaborative DevOps vision.
- Commitment is critical: This may also seem like a no-brainer, but the numbers bear out this requirement. Some 77% of Disruptors have agile development methodologies in place, with 70% having “broadly” implemented DevOps activities. For mainstream organizations, these metrics rate at only 31% and 21%, respectively; leaders are going all-in and appreciating the benefits, and those that follow will need to do the same.
- Experimentation is omnipresent: To garner the benefits of continuous delivery, embracing trial and error is a must. Among Disruptors, 93% recognized the need to “experiment and fail quickly on the road to success”, compared to only 58% of mainstream organizations. Some 95% of leaders have already shifted from fixed product cycles to continuous innovation, while only 56% of the mainstream have made this leap.
- APIs are everywhere: Ubiquitous integration, both in-house and in connecting to third parties, has only increased emphasis on the API. Disruptors are resoundingly focused on API management, with 68% using APIs to integrate third party services into their applications, and 66% to offer external access to systems and data. Mainstream peers compare at only 26% and 24%, respectively, denoting a sizable API management gap.
These are only a handful of the most dramatic and obvious areas where DevOps advocates have clearly re-architected their approach, separating themselves from those organizations that have yet to go all-in; it doesn’t stop there.

As highlighted in the research, other areas of differentiation and dichotomy include cross-functional alignment, emphasis on digital channels, use of digital to drive core business efficiency, and even funding of digital investments.

Beyond these key philosophical milestones that litter the pathway to DevOps success, the survey also highlights the vast array of strategic goals that these organizations have sought to accomplish. The results spotlight the massive range of significant business benefits that practitioners believe DevOps can deliver.

Leading use cases include:
- Improved customer satisfaction (49%)
- Better workforce productivity (44%)
- Improved customer experience (41%)
- Improved competitive differentiation (38%)
- Better agility / speed-to-market (37%)
- Opening up of new markets (35%)
- Creation of new business models (30%)
- Creation of new routes to market (26%)
If one were to take those outcomes out of context, unrelated to the impact of DevOps, it’s fair to say that they read like a generic list of leading business initiatives – further underlining the movement’s overall relevance.

That’s not to say this is all accomplished easily. Report respondents detail hard won victories that required significant time and investment to achieve all this DevOps-driven advancement. After all, the notion of “disruption” doesn’t exactly evoke images of ease or comfort.

Click here to read the report for yourself – there’s a lot more to be considered. Namely, how well your organization measures up in affecting digital transformation in its own right, and where it can take steps to move forward.
October 14, 2015
Native Integration of APM Driving DevOps Maturity

With applications performance monitoring established as one of the fundamental cornerstones of today’s DevOps methodologies, it’s easy to overlook the reality that effective instrumentation of such tools remains extremely difficult. Specifically, the biggest challenge remains creating performance metrics that address both IT and larger business initiatives.

There’s no question that APM is one of the most widely adopted and deeply embedded elements in today’s DevOps world, serving as the critical source of measurement that enables more transparent and agile applications. Yet, despite the fact that performance monitoring solutions, and their respective providers, are considered to be among the most mature on the DevOps landscape, customers still struggle to optimize the systems to meet their specific requirements.

However, as the market continues to advance, specifically via direct integration of widely-adopted APM tools with other strategic applications management systems, such as e-commerce platforms, organizations are finding a faster path to success, and bigger-picture DevOps benefits. One such partnership, announced in late August, saw market leader New Relic expanding integration with e-commerce platform provider Magento, owned by eBay.

Jon Azrielant, director of marketing at online retailer Jewelry.com, a Magento customer, said that his company initially ran into significant obstacles in adapting New Relic’s APM capabilities to its processes, despite the comparative strength of the tools.

“We had a really difficult time implementing New Relic initially, it was a painful process,” Azrielant said.

While the APM tools provided Jewelry.com with the data necessary to generate vital alerts, along with more agile sysadmin and operations monitoring, the more silo-based, fragmented approach that it employed could not empower the retailer to circumvent traditional walls between apps support and service support teams, Azrielant said.

Leveraging the extension of New Relic’s Software Analytics Cloud within Magento, the company has created more advanced instrumentation to inform critical business decisions. Among the most helpful of these measurements are those relating directly to customer experiences, while correlation of such metrics with Jewelry.com’s bottom line has also advanced executive decision-making, he said.

“The critical result of this integration is more complete visibility across applications and business performance, importantly tied to the actual business ramifications,” said Doug McIver, director of product for Magento. “In the past, merchants had to do a lot of difficult integration to use these systems together, so obviously this is a hugely valuable capability for us to be able to offer, especially to customers with highly performant site requirements.”

Moreover, Magento claims that it is growing its own customer base by tying APM more closely to its platform. Likewise, New Relic officials concede that to help advance crucial DevOps workflows, it must partner with such platforms to deliver APM data within the context of business requirements.

“If we’re talking about how to win in world where we’re iterating the end users’ experience on a regular basis, we have to put performance metrics within context of the UI, and the business results; working from the same data sets and getting people on the same page is incredibly important,” said Abner Germanow, senior director of solutions marketing at New Relic.

While pure performance stats are “near and dear” to New Relic for obvious reasons, the solutions provider is also attempting to advance a broader array of information that drives closer collaboration through new “iteration and experimentation”. New Relic is also focused on further developing the specific types of information most critical to key constituents such as e-commerce platform users. The need to create a common language for those DevOps efforts is amplified by the fact that development and operations officials are often unsure which metrics are critical to the business, and vice versa.

“Instrumenting code has always been a challenge, and we’ve worked very hard to ease that pain across a lot of different platforms; there’s always another language where work can be done, and we look to our customers to help lead us into the next phase of instrumentation,” Germanow said. “We’re seeing a lot of pressure around the need for constant change and iteration in industries where it’s truly a matter of survival, such as e-commerce, publishing, digital governance and finance – places where the industry is behaving very differently now than it has over the last 50 years.”

This work, and the subsequent APM integrations offered via platform providers such as Magento, have allowed end user organizations to remain focused on customer experience and sales, the ultimate goal of DevOps, said Jewelry.com’s Azrielant.

“In terms of how this relates to the DevOps revolution, this is the advancement that is bringing everything together and allowing everyone to look at the same numbers, to share common APIs, to streamline reporting and event response, and define everything in uniform terms,” Azrielant said. “Ostensibly, the most basic tenet of DevOps centers around the unification of these disciplines, and you can’t support that without the unification of this information.”

September 29, 2015
Steer SAFely – Driving collaboration across multi-disciplinary teams
This is the second blog in the DevOps – Scaled Agile series. In the first blog, Scaling Lean and Agile for the enterprise: All hands on keyboard!!!, we discussed the characteristics of a successful enterprise agile transformation and described why engagement and commitment across all disciplines in the organization is not just optional, but required, to ensure you deliver differentiated value. Business and IT must be in sync because, as we have learned through experience, successful agile team-based development does not by itself guarantee organizational success.

What does it mean to “be in sync”? How do we do it? Well, you Steer SAFely, of course. I know, a cute play on words, but just think about it for a moment. Consider a typical “day in your life”. From the time you wake up, you navigate through your day crossing paths with others along the way and continually adjusting in response. Your husband beats you to the shower, so you go get another cup of coffee. A guy crosses three lanes of freeway traffic (in front of you!) to get off at an exit he’s nearly missed, so you slow down a bit. You get a last-minute request to do something by the end of the day, so you engage with a colleague to get it done. Some of this is more reaction than collaboration, but nonetheless it is a simple analogy for Steer SAFely: Steer, which implies a series of continual adjustments to enable you to maintain course, and SAFely, which applies guidance and best practices as you Steer. The notion mirrors some of the basic concepts embraced by the IBM DevOps framework:
- Being aware of your surroundings (understanding dependencies)
- Recognizing and avoiding risks and roadblocks (monitoring and adjusting based on feedback)
- Working with others (collaborating)
Clear as mud? In a nutshell, to Steer SAFely means, as an enterprise, you are applying lean and agile principles to accelerate value delivery. To be successful requires collaboration of multiple disciplines across business and IT, continuously, from planning to development to deployment to delivery and then back to planning in response to feedback.

Let’s explore how you do this.

Step 1: Shift how you act and make decisions:
1. Focus on delighting the customer instead of making money. Making money is the result, it should not be the goal. This notion is specifically enforced in the SAFe Portfolio layer, which requires the organization to take an economic view to help guide business priorities based on an understanding of the value of something versus its cost.
2. Reconsider the role of bosses. Management’s role is to enable those doing the real work to be as efficient as possible by recognizing and removing impediments quickly, end-to-end. Agile used to mean “no management”, but enterprise agile is not the same. You need managers but their role is different. The SAFe tiered approach clearly describes the responsibilities and activities of managers at each level so that there is no confusion.
3. Avoid steering top-down or bottom-up. Neither of these methods leads to success. Instead, drive collaboration horizontally across the organization and involve all disciplines early and often in the decision-making process. SAFe explicitly addresses this notion through a set of well-defined events and activities that are designed to engage the entire multi-disciplinary team.
Step 2: Ensure the transformation completely addresses all aspects of the environment:
1. People: Establish an organization-wide culture with the right set of skills to embrace lean and agile values – everywhere, across all teams, every day
2. Process: Apply lean and agile principles in the planning, development, deployment and delivery of value and then monitoring feedback to improve – continuously
3. Tools: Provide a framework that enables the implementation of lean and agile practices – easily
The trick is to balance efficiency and effectiveness so that you can answer these questions in a positive way:
- Are we moving as fast as possible? Efficiency in execution is achieved through lean practices to minimize waste and maximize time in value added work.
- Are we driving in the right direction? Effectiveness in execution is the result of smarter steering by quantifying value and cost tradeoffs through continuous feedback and better delivery analytics.
Step 3: Adopt (at least) these three SAFe concepts:

Kanban System: Kanban embraces lean principles to ensure that you invest just enough at the appropriate phase of your lifecycle within your capacity to do so. SAFe describes a phased analysis approach bounded by Work in Progress (WIP) limits and grounded in economic thinking to quickly rank work based on not only value but also cost.

WSJF: Weighted-Shortest-Job-First is a value assigned to each of the key SAFe artifacts: Portfolio Epic, Program Epic and Feature. This value essentially defines an estimated return on investment and enables your team to consistently and relatively rank work across the Portfolio to drive priorities down to the Teams. Use of WSJF enforces the economic view in planning because it ensures that both value and cost are considered as you prioritize work.

PI Objective: SAFe describes a Program Increment (PI) Objective artifact at both the Program and Team levels as a way to document expected and realized value in a relative way. Here, as with WSJF, consistency and relativity are most important, not precision. PI Objectives are also a way to capture the notion of acceptance criteria, which could guide solution-level validation.

To learn more about our thoughts on SAFe and DevOps, try this: SAFe® with the Power of IBM DevOps.

And please join IBM at AgileDC, October 26, 2015, Washington D.C. Submit a proposal through August 15, 2015. URL: http://agiledc.org/2015-call-for-proposals/

About the Author/ Amy Silberbauer

Amy is a Solution Architect charged with the definition and delivery of the Enterprise Scaled Agile and IBM DevOps Plan solutions and strategy for the IBM Systems – Application Platform line of business. She is a recognized subject matter expert on scaled agile and software development lifecycle solutions, including enterprise modernization, SOA and collaborative DevOps. She has been with IBM for 28 years and has two decades of software development experience as an engineer, architect and manager. She is a certified SAFe® Program Consultant experienced in leading and consulting on multiple internal and external SAFe (Scaled Agile Framework®) transformations.

Connect with Amy on
August 11, 2015
Librato Seeks to Unlock DevOps Application Monitoring

As wider adoption of DevOps methodologies continues to foster demand for new tooling to help streamline evolving workflows, at least one vendor is attempting to address the growing need for related monitoring capabilities.

Librato, a San Francisco-based provider of Amazon AWS applications performance monitoring (APM) services, claims that it is finding favor among the rapidly expanding DevOps community driven by its ability to inject critical visibility into the engineering lifecycle.

Acquired by Austin, Texas-based SolarWinds in January for a lofty $40 million and renamed Librato SolarWinds Cloud, the company is touting its recent adoption by well-known providers such as Slack for monitoring of DevOps-centric applications performance and health metrics.

“At its core, DevOps is destroying conventional wisdom around ritualized processes and the need to perform them, mostly in the domain of scale,” said Dave Josephsen, Developer Evangelist at Librato. “By attempting to include monitoring as a key element of the development lifecycle, this has created a whole new range of challenges, and opportunities.”

By providing a centralized Web interface and open API to help create visibility into critical applications engineering requirements, Librato maintains that it is creating the tooling necessary to empower integrated development and deployment processes, such as continuous change control.

“There are a lot of disparate tools for monitoring individual engineering tasks, but there hasn’t been any practical way to bubble up that data for efficient monitoring,” Josephsen said. “Within the context of DevOps, change control is a ritual that we’ve abandoned as we’ve embraced test driven development and automated away the human element, so there’s a distinct need for monitoring of feedback and metrics.”

For example, cloud-based productivity messaging specialists Slack are using the Librato “StatsD” lightweight daemon process to stream business and app-health metrics regarding PHP applications into the monitoring platform. Slack also leverages Librato to chart data regarding connect, query and fetch latencies for its server clusters, and to monitor internal metrics about overall system health.

“Having those graphs side-by-side lets us clearly see and compare the effects of network latency on one of our new services, and that’s very cool,” Richard Crowley, director of operations at Slack, wrote in a recent testimonial posted to the Librato Solarwinds blog.

Industry experts agree that broader adoption of DevOps strategies will continue to drive demand for relevant monitoring tools, with solutions such as Librato seeking to deliver “modern stack architecture for looking at the modern stack.”

“I think that APM is always interesting because it’s always new,” said James Governor, industry analyst with RedMonk. “The pace with which we’re building new architectures to deliver services to end users is increasing all the time, and with every new wave of technology, we tend to get new and associated models of APM tooling.”

Librato is taking advantage of the need for a system that brings together the elements necessary for cloud native monitoring, and in an agentless manner different from other competing tools such as New Relic, the analyst noted.

However, whether or not the platform is able to break down traditional engineering siloes to enable more efficient DevOps management, as Librato officials claim, remains to be seen, he said.

“Each wave of technology has promised to do this, to remove the inherent finger pointing, and each has always made the same claim to break down organizational firewalls,” Governor said. “DevOps hasn’t boiled the ocean in this manner yet, but there are some unique opportunities; unlike previous waves, where automation was intended to replace humans, this is now about augmenting people, and we are seeing that teams are choosing new tools from which developers can truly derive value.”

August 10, 2015
Surprise! Broad Agreement on the Definition of DevOps
For the past few years, the conventional wisdom has been that DevOps has no agreed upon definition. Instead, everyone has their own interpretation. However, the conventional wisdom is wrong.

While everyone supplies slightly different words, the same working understanding of DevOps is nearly universal. Our understanding of DevOps goes back to a handful of common touch-points. There is the “CAMS” (Culture, Automation, Measurement and Sharing) acronym popularized by John Willis and Damon Edwards. CAMS has been extended repeatedly – Eveline Oehrlich of Forrester is now discussing CALMSS. Gene Kim offers his “3 Ways” and Patrick Debois has codified “4 Areas”. Allspaw and Hammond had 10 deploys a day, and Etsy made technologists fascinated by a virtual crafts bazaar.

Analysts, authors, industry, and community leaders all agree. The ideas are big and mutually supportive. In short:
- DevOps exists to help the business win
- The scope is broad, but centered on IT
- The foundations are found in Agile and Lean.
- Culture is very important
- Feedback is fuel for innovation
- Automation helps
Why: To help the business win

The most popular DevOps book today is The Phoenix Project, with a subtitle “A Novel about IT, DevOps and Helping Your Business Win”. DevOps is not a navel gazing IT methodology for making IT better for IT. It is focused on how stuff involving the technologists can deliver better results for the business.

As Patrick Debois writes in his excellent post,”DevOps Areas – Codifying devops practices“:

As rightly pointed out by Damon Edwards, devops is not about a technology, devops is about a business problem. The theory of Constraints tells us to optimize the whole and not the individual ‘silos’. For me that whole is the business to customer problem, or in lean speaks, the whole value chain.

At IBM, our definition of DevOps calls out enabling the business to “seize market opportunities”.

Today, technology is critically important to how a wide variety of companies compete. Retailers who neglected e-commerce have gone out of business. Logistics companies like UPS find a competitive edge by creating software to shave miles off delivery routes. In the automotive space, Ford is at tech conferences recruiting companies to write apps for its cars and Tesla’s first fix for their flammable battery problem was an over the air software update.

DevOps is a response to what the IT is experiencing – from a back-office concern that needed to “keep the lights on” to key part in how the business competes every day.

Who: A Broad Scope, beyond development and operations.

Yes, DevOps is a portmanteau that brings development and operations together. Yes, the classic DevOps concern is the Wall of Confusion between development and operations. However, with the focus on business value, the scope must include everyone who is involved in taking an idea through IT to business value.

In recent years, there’s been something of a cottage industry of community leaders and analysts writing, “No, no it really should be Dev_____Ops” where the blank is filled in by their own specialty. Examples: DevQAOps, DevOpsSec, DevSecOps, BizDevOps and of course Bussdevtestqanetsecnetops.

When a Finance department must approve the provisioning of each new test environment, slowing the delivery of new applications by weeks or months, there is a DevOps problem that leaves narrow confines of development and operations. It leaves IT. Patrick’s quote above talks about optimizing the whole value chain. In that post he describes DevOps as being about “collaboration, optimization across the whole organization. Even beyond IT (HR, Finance…) and company borders (Suppliers)”.

Based On: Agile and Lean

In answering why he didn’t support a manifesto for DevOps, Patrick Debois points people back to the Agile manifesto and asks them to reread it more broadly. Agile’s shift towards a faster delivery of software, and focus on the end goals make it a natural spiritual ancestor for DevOps. However, from the perspective of Ops teams, Agile represented the barbarians at the gates. Agile generally took root in development side of the house, enabling it to create something the business wanted faster. But with operations still moving at a pace suited to waterfall, pressures grew and the need for something like DevOps became clear.

Where much of DevOps energy seems to come from its Agile roots, the intellectual side of the movement is more rooted in Lean. It seems no accident that many of the Phoenix Project’s lessons are taught away from the cube farm and instead from the factory floor. While we can intuit that faster feedback is better, and the “continuous” in CI and CD is convenient because it lets us map feedback to fewer changes, it is in the Lean literature where we encounter Little’s Law. It provides the queue theory mathematics supporting the idea that smaller batch sizes supports shorter cycle time. And so Reinertsen’s “The Principals of Product Development Flow: 2nd Generation Lean Production Development” has become popular in DevOps circles and is well cited by Jez Humble in his new Lean Enterprise book.

Culture: Collaboration and Experimentation

The Agile and Lean roots show through in the DevOps culture as well. Lean and Agile are both focused on people first, systems second and heavily cross-train or work in cross-functional teams. They also emphasize continual improvement (kaizen) through approaches like retrospectives.

In DevOps circles, collaboration is preached as an antidote to thinking in siloes which has resulted in antagonism between development and operations. For the systems thinking required to deliver for the business, teamwork is required. Developers make better decisions when they take advantage of operations knowledge (Patrick’s Area 3) and operations does better when application knowledge is embedded with them (Area 4).

The other cultural emphasis is a push towards experimentation, constant learning and improvement. This is Gene Kim’s third way of DevOps. The Lean Startup ideas of Eric Reis permeate the DevOps world. Intuit has made great strides in showing how these approaches work in established companies by running experiments in the midst of tax season resulting in improved business outcomes.

These cultural trends come together in things like blameless postmortems discussed by Etsy and Jake Loomis in Web Operations. When things go wrong, organizations need to understand why and take corrective action. In troubled corporate cultures, the messenger reporting the problem might be shot, and the hunt to assign blame for the problem will begin. Healthy organizations look to learn from failure and try to improve the systems that allowed (or guided) people to make a series of mistakes that lead to the failure.

Feedback: Yes. Lots please!

Experimentation, adaptation and learning tend to do better when there is ample feedback. Otherwise, there is too little to learn from. Developers tend to be more productive as well with rapid feedback. Their thirst to know whether the latest change diminished quality led to the practice of continuous integration and the tools that support CI and continuous delivery. That CD ends up facilitating faster delivery of value to the customer is something of a happy accident.

Amplifying feedback loops is Gene’s “second way“. The “CAMS” concepts of measurement and sharing point to feedback. You gather lots of information, and make sure people see it so they have an opportunity to learn from it. And when Elisabeth Hendrickson presents at DevOps Enterprise 2014, as Cloud Foundry’s Director of Quality Engineering at Pivotal, she describes her job as “to see to the care and feeding of the feedback cycles“.

As expected with a collaborative culture with ample sharing, feedback must cross the traditional silo lines. That means the awesome production monitoring that had lived in Ops land, is radiated back to development. This is “Area 2” in Patrick Debois codification.

Automation: Of course.

The “A” in CAMS is automation. Patrick’s “Area 1” points to continuous delivery techniques, well described by Dave Farley and Jez Humble in their book. A Lean initiative looking at IT will find waste in wait times, errors, and more and move towards automation as a natural result. Feedback craves automated deployment of changes, and automated tests or production monitoring that validates them. Automation is what makes a lot of DevOps possible. It doesn’t make cooperation between development and operations happen, but tensions are reduced when Ops gets out of the business of arguing about the SLA they have with Dev around server provisioning and instead provide Dev a button to press.

DevOps isn’t about automation. Automation is just the natural result of DevOps.

Additional Resources not cited above

DevOps for Dummies – by Sanjeev Sharma

8 Best Practices for DevOps – Ashok Reddy’s overview of the Forrester Report

About the Author/ Eric Minick

Eric Minick is a Technical Evangelist. He joined IBM through its acquisition of UrbanCode. He has spent the last ten years helping organizations – large and small – adopt continuous integration, delivery and now –DevOps. He is a frequent speaker on the topic, and co-authored Application Release and Deployment For Dummies.

Prior to consulting, Eric was a developer, test automation engineer and support engineer, and has contributed to multiple generations of UrbanCode products. Follow him on Twitter at @ericminick.
May 13, 2015
The Last One That Transformed the Future
The headline made her yelp for joy. A year into the future of D-Chip, on a crisp August morning, Phoebe opened the newspaper and swiped to the technology section on her iPad. What she saw was riveting: D-Chip reborn as a star in wearable computing.

She immediately thought of the article Marc Andreessen wrote on how software transformed the enterprise. It’s true of our company’s story, she mused. The credit for making the headline a reality was due to the stressful events of the last months. Who said stress couldn’t be a game changer? As she picked absently on a plate of fruit and relished the article for the second time, she recalled the events of the past year.

Flashback a year ago at D-Chip

Remember how Rachel pitched her enthusiastic idea to Chandler? She wanted to engage six engineers in six months to build an in-house solution. A solution that would integrate D-Chip and Enogo’s systems by letting them deploy in both OpenStack and AWS and manage wearable chip code from a single deployment platform.

Chandler agreed in theory with the need for a single management platform but didn’t see the sense of doing it in-house. Why side-track from your business goals and spend months building a solution yourself when there are specialists solving this exact problem for many companies facing the same issues? After much debate, they came to a consensus. They decided to evaluate solutions from the best-of-breed products out there and transform their software development lifecycle to reflect the best agile practices. Thus was born the iTransform project.

The iTransform project

The name itself was a reminder that transforming was as much a matter of people internalizing and accepting a new way of doing things as much as it was about aligning and unifying D-Chip and Enogo systems. The first order of priority was to understand the pain-points that assailed the company. After a lot of meetings and detailed analysis, they identified key use cases from internal stakeholders, namely the engineering and IT operation teams:
- Engineering personas, developers, QA, architects, all said the same things. They didn’t have access to the latest tech because it took weeks or months for approval processes to travel up the chain. Then they waited on builds and environments from IT operation teams. Most of the time, the environments in the private datacenter OpenStack cloud didn’t work properly, and deployments never succeeded the first time. It’s no wonder they took weeks or months to release anything decent.
  
  Self-service and continuous delivery were, therefore, the most important criteria for developers, QA, and architects. Without waiting for IT admins or operations teams, they wanted to spin up build environments in test, staging, even production on their choice of pre-configured infrastructure. They wanted to consume in an on-demand fashion the latest technologies on multiple clouds. Today they were using AWS cloud and OpenStack, but tomorrow they wanted any cloud that matched their workload configuration needs best.
- IT operations and system admins brought up their set of challenges. Their primary goal was to configure environments for repeatability; safeguard builds from failures and structure stable workflows. Not an easy job come to think of it. On top of this demands from the engineering teams dictated that they learn and implement new technologies in a very short time. Technologies ranged from configuration management tools, cloud platforms, storage, networking, load balancing services, monitoring services, and more. Besides when engineers wrote configuration templates as Enogo did for AWS, it was hard to figure out who owned the deployment stack. In the case of problems, it resulted in pointless blame games. So they desired a clear level of separation between code, deployment automation, and infrastructure. Technologies and cloud platforms could come and go, but IT operations remained responsible to maintain smooth workflows, stable builds, and error-free deployments.
  
  A service catalog also met the needs of the IT operations teams. They wanted to build and manage specific platforms, runtimes and version configuration changes. Collaboration was a key factor. Many IT operations team members and engineers were experts in some but not all technologies and tools. But together in a collaborative platform, experts could contribute at their skill level and share knowledge with others. Automating configuration changes in bulk saved a ton of time. Therefore, API support for automation was high on their wish-list.
- The IT managers were important though indirect stakeholders. They often approved and handled audits, compliance, and budgetary aspects. Policies and cost management were paramount to them. IT managers needed control and oversight to implement compliant deployment policies that gave the right level of access to the right set of infrastructure on pre-approved cloud providers. They needed to set and monitor quotas and track IT spend on cloud deployments against approved budget.
Transformed enterprise

After three months of evaluating proof-of-concept solutions with vendors, they zeroed in on a solution that met all of their uses cases and helped set up their current builds. Because of this, the teams were able to ship code ten times faster. It was only a matter of time before D-Chip stood out as a market leader. It was clear to the company executives what was driving this transformation.

They retired the semiconductor business and switched their business focus entirely to SDK builds of software for wearable computing. The software services for wearable chips was now way more valuable than any piece of hardware. Their customers in wearable computing now consumed D-Chip technology as a service in the cloud. A faster and efficient software development lifecycle enabled them to carve a new market niche and become the de facto leader.

The transforming story of D-Chip is a reminder for us all. When there are problems in software development, by aligning goals and helping people in different roles work together and participate in the development lifecycle, you increase the pace of innovation dramatically. It can be so powerful that it can transform the company’s core products or services.

Disruptive change doesn’t happen one step at a time. It happens with mavericks and innovators like Rachel and Chandler leading radical changes in a focused group effort. You want innovation to transform your business? Get a dedicated group of mavericks together and make it happen.

Just like mankind advanced their understanding through each cycle of civilization from the stone age up to the Renaissance and the Industrial Revolution, the future is heralded by the IT age of our times.
April 6, 2015
DevOps & Continuous Change

A remark by a colleague while waiting for the coffee machine to complete its cycle started my train of thought. “Should we have multiple minor releases or just do a few major ones in a year?”

In large organizations, due to many factors, the turnaround time for a single successful release is quite extensive; but if the conceptual change which we are talking about bringing whilst using the #DEVOPS methodology would not only reduce these long cycles but deliver quicker and quality software.

I have already discussed in my earlier article “DEVOPS: Getting our organizations aligned” on the traditional approach and how the combination of this two (DEV & OPS) has been proven beneficial in various organizations and how many are still hesitant in embarking on that path.

Continuous Change which encompasses both Continuous Integration and Delivery is paramount to achieving this objective. To attain the objective of delivering rapid and consistent value to the stakeholders regularly, we have to have Continuous Change happening.

I am sure we have already moved out from the Mainframe era to a more Distributed systems state. But unfortunately we have yet to let go of that culture where we pile up components on a single quasi Distributed Application, instead of creating smaller independent, yet interdependent systems.

So, back to the same question; shall we go with a few Dinosaur sized, labored releases or make multiple short and swifter runs? In my opinion, we should strive towards the leaner model. Easier said than done you say! Agree. If I have to, then I would go about it this way:

In an environment having a suite of Distributed Applications, start with Analyzing the components which make up the Major release, List out the changes (fixes and new features) which are planned, asses functional and technological impact, Identify dependencies, upstream/downstream connectivity’s, architectural deficiencies, list out the identified subset of components or group of components by functionality, which can go as an independent release.

I am resisting the urge to say ‘standalone’ as with the current set of complex intervened collection of applications, it would be an incorrect statement. It would rather be a subset of components which can be independently upgraded. Now the big task is to review and decide if this subset or group would add functional value to the overall suite of applications without the rest of components going in.

Once identified, the success of this release working as expected would be in the extensive testing (Unit/ Regression/Performance) of these components, and as we gain confidence in this split and deploy procedure, we can start with, prioritizing and scheduling critical must have features or bug fixes which also should be backward compatible with other components lagging behind. We can even come up with scenarios where these components are not just getting updated frequently, but would also have multiple independent tracks. Of course it goes without saying the testing and signoff process is strictly followed albeit in a shorter window.

Deployments to production have always been a source of heartburn to the OPS teams, we have often seen finger pointing and firefights to contain change fails. In order to maintain stability and maximize availability, stricter controls and reduced deployment windows are put in place. In promoting frequent and smaller component level changes, OPS will also be in control of what’s going in, and smaller component level releases can be rolled back quickly within the Green-Zone window if the change is not working as expected. Though I would categorize this rollback, as a process fail than a deployment fail, and should be reviewed very seriously with right mitigation plan and lessons learnt retrospection, Continuous learning after every successful or failed sprint ensures competent and confident releases.

Another major factor impacting multiple runs of Production releases is the long Green-Zone windows, by moving to the quicker and leaner deployments cycles, these requests can be substantially reduced, and can eventually aim to reach a stage where changes are deployed online without extensive downtime.

Similarly, frequent maintenance activities when there are no planned deployments, effectively has the same risk on environment unavailability. Analyzing and moving to a state where these activities are conducted online without a Green-Zone would alleviate the maintenance downtimes and ensure seamless availability, adding stakeholder value.

I have not touched on the Continuous Integration or the Automation of the deployment process for obvious reasons, we can’t achieve the objective of Continuous Change without having the Integration process in place where incremental changes to code are build into a package swiftly and seamlessly, even a sanity or basic testing might be integrated to ensure the quality of the builds. Same goes for automating the deployments, develop & integrate tools required for rapid deployments. We have seen instances where automating a single activity or revisiting a process has created substantial time savings.

While it’s easy to propose changes to the Release Process or advocate rapid deployment cycles, we have to ensure adequate checks and balances are in place, compliance adherence and audit trails are created.

Now to the most important measure of this article, Cost Savings!

Proponents of the Dinosaur Releases would argue that the common activities which entail a single release like the builds, Testing, Release Management, OPS availability or reduced GZ would cut costs and multiple runs to PROD would logically increase costs.

But it’s definitely the other way around, we wouldn’t need to feed the Dino anymore, it will be a cost worthy rapid delivery with optimized resources, a leaner process where DEV and OPS are working together to ensure quality value added deliverables are made available to business faster.. This cultural change and collaborative effort also creates new vistas of growth and provides a perfect platform to excel. The cost benefit analysis would eventually be in favor of Continuous Delivery.

Opportunity cost is another variable which has not been considered; an innovation or a new feature available to customers quickly with reduced time to market cycle will provide the much needed edge in these competitive times. I recall some conversations where a few must have’s were pushed out due to various reasons, frustrating the business.

DEVOPS was never meant to replace the traditional approach to software development; rather it is an efficient use of the existing resources within a collaborative model to rapidly deliver quality software products. A major cultural shift in our approach is required and to reach that goal we need to embrace the shades of grays, many a times over 50 (sorry, couldn’t resist) when it’s not truly black or white.

Thank you for the time.

Visit my blog http://askhurram.com/

Read On, Live On.

February 24, 2015
Four strategies for supporting DevOps collaboration

As we’ve discussed before, the ever elusive DevOps culture doesn’t form magically. Rather, it is fostered by building a collaborative environment. While there are many ways to skin the cat in terms of what kind of tools organizations use to start collaborating more effectively, the tool set and the process they enable should revolve around some key principles. The following strategies can help guide organizations to choose the right infrastructure to foster a transformative DevOps culture and support DevOps collaboration.

Centralize And Unlock Knowledge

One big mistake some organizations make as they embrace DevOps is they consider their automation tools as knowledge repositories. But things like recipes, manifests and YAML files are great for automation, not collaboration, explains Alan Sharp-Paul, CEO of ScriptRock.

“They are an awful way of storing desired state,” he says. “Desired state should be visible. It should be easy to understand and simple to modify by any stakeholder. If the systems that define your desired state are not open and accessible then collaboration will suffer. When knowledge is locked up, collaboration breaks down.”

Sure, there may not be a super platform for everything, but centralize documentation with pointers out to the right details, says Chris Clarke, senior vice president of product management for CollabNet, who suggests that this place should show how work is spawned and what is being asked of whom, where work is being outputted, where to find all of the tools being used across the development and deployment lifecycle and visualizations, diagrams and step-by-step details about important processes.

“It’s almost like a mall map. ‘You are here,’” Clarke says.

Enable Timely Communication

Of course, knowledge doesn’t stand still and there are always day-to-day snags that come up, so constant communication is fundamental to the collaborative process. Many DevOps and Agile leaders find the success they see in speeding up iterations stems around the experimentation they’ve done in developing a communication channel that work for their teams.

This usually starts with using a chat platform like HipChat, FlowDock, or Slack, and frequently continues by pairing that with task managers like Trello, Asana or Basecamp. Many organizations are also integrating IT-specific tools like Git, PagerDuty and Raygun into this mix.

And, increasingly, DevOps teams have been up-leveling that by implementing GitHub’s open source side project, Hubot, a chat room automation bot that helps teams share scripts, automate tasks and enable simple deployment tasks and keep everyone else apprised of deployment.

The sky is the limit for creative combinations—the key is that the overall stack facilitate timely and meaningful communication across the team.

Find Ways To Visualize Technical And Process-Based Interdependence

So, what about SharePoint to accomplish these goals? The problem with SharePoint, says Matt Selheimer, chief technical evangelist for ITinvolve, is that it really isn’t designed around the workflow and the dependencies of IT-specific work. As he explains, most organizations never get beyond the stage of using SharePoint as a glorified file share.

“That’s not going to help you understand risks and relationships and dependencies and nobody has time to go and read through all those documents and try and parse them anyway,” he says, explaining that a collaboration platform should be able to help various employees track activity around applications, policies, servers and what have you that they have a stake in. When someone else is touching those specific objects, they should be alerted and connected with the resources necessary to look into potential issues, resolve any problems that arise and potentially suggestion improvements on current progress.

Componentize Knowledge

Centralization is important, but that doesn’t mean that knowledge should necessarily be consolidated.

“You need to componentized it correctly—in other words, you don’t try to write a 40-page document, you use a wiki or something that should change pretty frequently,” Clarke says. “I’m a huge fan of continuous improvement in everything, including this.”

Selheimer agrees.

“We don’t think that trying to capture everything in one giant, massive document and have it be reviewed by a committee is effective because it’s never going to get published It’s going to be outdated,” he says. “Whenever I ask people about what diagrams do they have to use to manage their environment, they always say, ‘Oh, we got Vizios.’ But they’re all six months out of date, right?”

December 15, 2014
Detours Ahead: Overcoming DevOps Challenges

Saying that you’re interested in DevOps is another way of saying you’re interested in change. That’s what DevOps promises any organization. No matter the problems a company is trying to solve or the goals it’s trying to reach, DevOps depends on a willingness to embrace some big changes. Sure, it can help you streamline software deployments, but it requires shaking up a company’s usual processes and routines. As we all know, change can be complicated, so before your organization transitions to an agile culture, it’s best to prepare for the likely challenges ahead.

Breaking Old Habits for Continuous Delivery

Many of the procedures we follow at work each day are second nature, and what’s familiar carries a certain comfort with it. Even when we realize a more efficient process is within reach, old habits are tough to break. For an organization that employs hundreds, any change can be a challenge.

And DevOps offers no shortage of challenges. It entails overhauling a company’s deployment process, its communication patterns, its department structures and even the roles of individuals on a team. Yet unless an organization is willing to let go of its old habits, the benefits of continuous delivery won’t ever be achieved. When acclimating people to this new procedure, it’s important to convey that the old procedure – one team throwing code over the wall to another one – will be left behind for actual collaboration in which Development, Operations and possibly other departments work together, not just in silos.

Getting Past the Resistance to Change

Encountering some reluctance to change will be natural, and a smooth transition depends on making everyone aware of why DevOps will be a big step forward. This includes communicating what the advantages will be for individuals, departments and the company as a whole. With an agile plan in place, greater efficiencies will become the norm, and a team will have the ability to spot and fix problems more quickly, so 3 a.m. phone calls to the system admin won’t happen anymore. Once managers through C-level folks are aware of these potential advantages, resistance will gradually turn into acceptance and enthusiasm.

Combating the Myths and Rumors

Many think DevOps is only for the well-known unicorn companies, such as Etsy and Facebook, or that it’s an exact science that will produce the same results for everyone. Some even believe the rumor that DevOps only works with certain operating systems. The truth is that continuous development can take place anywhere as long as the company culture is right. As detailed in the Rackspace-sponsored e-book, The DevOps Mindset: Real-World Insights from Tech Leaders, many DevOps practitioners went through their own uphill climbs to become more agile, and they have the firsthand experience to resolve many popular misconceptions.

“You don’t have to be in Silicon Valley to embrace the DevOps culture,” says James Kenigsberg of 2U. “In fact, many new firms these days have a bad habit of building shiny technology because it is shiny, which goes against the DevOps principles of communication and transparency.”

Jeff Hackert of Chef dispels the myth that DevOps is a one-size-fits-all solution. “It is not a set of proscriptions or a recipe that will give you a defined best practices outcome,” he explains. “Your best practice is the collective wisdom of the very smart people you’ve hired to solve your problems. And if what you have is a siloed or broken environment, you want to increase the communication, not impose new rules.”

Getting Automated Without a Struggle

When an organization is considering DevOps, they may find that the opportunities for efficiency not only involve employee collaboration, but technology as well. Most IT departments would jump at any chance to save time when it comes to coding procedures, and automation is a means of making this happen. By offloading these processes to an automated system, people can tackle other tasks, and businesses that release several deployments a day can greatly simplify their rollouts. For those who might not be familiar with the ins and outs of automation, receiving some guidance from a trusted IT provider can be a good first step. Finding an advisor to guide your initial automation efforts can prevent the expense of hiring a DevOps engineer with automation knowledge.

Finding a Path Around DevOps Obstacles

Every company’s DevOps strategy will be unique, and every company’s plan for overcoming challenges will be just as unique. Whatever the circumstances, communication will be crucial to any adoption, as it ensures that everyone understands why such a change is necessary. Getting acquainted with the success stories out there will dispel any myths circulating among team members, and stumbling blocks can be avoided by the understanding that familiar silos will be replaced by more efficient collaborations.

Despite its challenges, DevOps can be the catalyst for a variety of improvements, from productivity to employee morale. Those who are the most successful at being agile will not only have the right culture in place, but be able to negotiate the challenges that DevOps has in store for those ready to embrace change.

November 19, 2014
Collaborate more effectively with AppDynamics DevOps Virtual War Room

DevOps enables companies to move faster—but moving faster is not a worthy goal in and of itself. Organizations have to be able to develop and deploy solutions that are effective as well as expedient. It’s important to be able to monitor and respond to events in real time to ensure everything is running optimally, and to be able to create and implement fixes as quickly as possible when it’s not.

At its AppSphere 2014 conference held in Las Vegas earlier this month, AppDynamics announced the latest release of the AppDynamics Application Intelligence platform—which is designed to help make application intelligence available so businesses can run faster, leaner, and more efficient. One unique element of the latest release is a new DevOps Virtual War Room.

An AppDynamics press release describes the DevOps Virtual War Room. “It enables IT operations, development teams and business users to see and resolve problems together in a shared virtual space. The new release also enables scheduled, auto-delivered reports, and adds a new iOS app that sends push notifications to team members when system alerts are triggered.”

Unless you can get all parties in one room, reviewing the same information, and discussing the issues and potential solutions, troubleshooting can easily devolve into chaos. Mixing various forms of communication—phone calls, emails, instant messaging, face-to-face, etc.—results in a modern day business form of the children’s game “telephone”. Statements get twisted, and bits and pieces of information get lost in translation as various parties attempt to work together.

The DevOps Virtual War Room strives to change that dynamic by giving teams—even geographically separated teams—a single virtual space where all parties can come together to collaborate. Everyone sees the same data, and the individuals can discuss options and make changes to application settings and configurations together in real-time.

Another benefit of the DevOps Virtual War Room is that it maintains a log of the communications and the changes that have been made. Valuable information about the root cause of the issue, the troubleshooting process, and the changes made to implement an effective solution are captured so that there is an audit trail to refer to if additional problems arise, as well as a historical archive that can server as a reference for future issues. The data collected from the DevOps Virtual War Room can be captured and indexed in knowledge management systems in order to streamline access in the future.

“We’ve all seen the tremendously positive impact the DevOps phenomenon has had. With our new Virtual War Room and the other tools introduced today, we are catalyzing DevOps collaboration for troubleshooting, as well as application development, in ways that no other APM platform can do,” explained AppDynamics CEO Jyoti Bansal. “Performance isn’t just about technology, as critical as that is—it’s also very much about people working together closely to get things done faster and more effectively.”

To be fair, there are a variety of other ways to accomplish a similar goal. Any number of Web conferencing or virtual whiteboard tools—including things like Skype for Business or Google Hangouts—could serve as a virtual meeting places where developers, operations, business stakeholders, and others could “meet” virtually to get things done. One aspect that makes the AppDynamics DevOps Virtual War Room different—if not better—is that it also brings the relevant data and information to the table, and presents it in a way that enables the individuals to analyze the problem and devise a solution.

November 13, 2014