devopsy.kinsta.cloud

Category: Enterprise DevOps

Mapping the Manager’s Journey to DevOps and Continuous Delivery in the Enterprise

Software development methodologies like DevOps and Continuous Delivery continue their rise to household name status. This is due in part to the increased mainstream media attention they get, but also from the diverse range of enterprise organizations like Target, who are announcing large investments in it, to GE who has shared one of their division’s success story with the world.

“The journey of a thousand miles begins with a single step.”

Daoist Chinese philosophy is probably not the first thing you would associate with improving software delivery at your organization. But in reality, it rings true as enterprise IT managers begin the daunting task of transforming the software delivery process. So why set out on this path?

One reason why IT organizations are moving to Continuous Delivery is to gain the ability to provide timely user feedback to the developers so software can better meet their demands. Gary Gruver, author of the enterprise IT best seller, A Practical Approach to Large-Scale Agile Development: How HP Transformed LaserJet FutureSmart Firmware and the former Director of LaserJet Firmware at Hewlett Packard lead a DevOps transformation in 2008, back when this portmanteau was barely on anyone’s radar. Up until that point, the group—comprised of about 800 developers globally—was averaging two releases a year. The vast majority of the group’s resources was swallowed up by the porting of existing code to new firmware and the process of check- it was taking up to eight weeks for a developer to learn whether their code change worked or not! The lag time introduced in the process of fixing those bugs was obviously a big detriment to their productivity and it prevented the developers learning what mistakes they might be making.

Continuous Delivery is about continuous improvement—it’s taking one step at a time on this journey that ultimately gets you where you need to be. In an interview with Alan Shimel, editor of devopsy.kinsta.cloud, earlier this year, he said “each organization is different. You really need to start where the most pain is. And one of the things that DevOps causes you to do is do things more frequently. And what you’ll find is your organization has been brute forcing its way through the same issues for years. And as you start to increase the frequency, you’ll have to start fixing that.”

Each organization is obviously different. For executives who have a better sense of the big picture and understand less of the day-to-day details, it’s important to have a clear picture of the existing development processes before beginning.

But, how do you translate these process details into something that improves at the scale necessary for the enterprise? Are there metrics that are important to focus on along the way? What type of team structure do you need in place? What’s the best process? Kicking off a DevOps transformation takes some effort but companies who embark upon this are seeing notable results. Fortunately, Gary and Tommy Mouser, Director of Test Automation at Macys.com can help make your journey easier. They have distilled their real-world knowledge of doing this in their new book, Leading the Transformation: Applying Agile and DevOps Principles at Scale.

On November 17th, my colleague Andrew Phillips, Vice President of Product and active developer/DevOps evangelist in his own right, along with Gary and Tommy will be sharing their experience and practical advice for kicking off your own DevOps/Continuous Delivery journey. It’s an interactive panel webinar so questions are strongly encouraged! As an added bonus, just for attending you will receive the first 2 chapters from Gary and Tommy’s new book. And for those with really interesting questions, we’ll send you the whole book! I look forward to you joining us for a lively discussion on the 17th!

Register Today!

About the Author/Heather Moses, Vice President of Marketing

Heather heads marketing for XebiaLabs. In this role, Heather drives global marketing strategy and operations for the Company. As part of the senior leadership team, Heather is responsible for market direction, brand evolution and market outreach best practices.

November 2, 2015
5 Tips for Building a Winning DevOps Culture
Despite our cognitive prowess, humans are just like other animal in the food chain. We follow patterns of behavior based on survival. If food is safe to eat today, we’ll eat it tomorrow. If we can navigate a road without getting hurt, we’ll do it again and again.

This explains why change is so hard. Evolutionary magic has taught us to avoid stuff that could harm or kill us – in life and even in business.

But the business jungle is changing faster and far more radically than ever before. The pace and ubiquity of technology has resulted in once sturdy business models being brutally felled. Now we must move out of our comfort zones – or get eaten alive.

Businesses now require regular and sustained innovation, but this is problematic for many organizations more used to operating in safe mode. For them, fear of change reigns supreme – even when the company is losing market share and talent is leaving in droves.

The situation is recoverable when leadership commits to building culture of innovation. This is especially important in IT, because it’s here where innovation now starts and ends. To this end, organizations are rushing to embrace Agile and DevOps – the practices of choice for rapidly delivering high quality software.

But when it comes to culture nothing is easy. Here are five tips to help ensure success:
1. Avoid Building Elitist DevOps Teams – the natural tendency with any new initiative is to build out a separate team. This means bringing in outside experts, cherry picking what’s seen as the best internal talent across development and operations, and then operating distinctly from the rest of IT.
It must be remembered that DevOps isn’t a function, but a model in which development and operations collaborate. Therefore building a separate team is just creating another silo, which because of the “elitist” association could create even more friction.

DevOps teams can valuable during early stages of a program, but only then as a transitional practice. In these case members of the team are chartered to spread knowledge; building the bridges across the organization and then disbanding once practices start to take hold.
1. Cull Elitists and Destructive Narcissists – a strong DevOps culture is based on collaboration and empathy; sharing success and learning from failures. If, however, initiatives are led by individuals who strive for personal glory and kudos at the expense of everything – especially staff morale and wellbeing, then there’s a problem.
Doubly difficult is the fact that people in this category are energetic extroverts; appearing ideally suited to spearheading important cultural change initiatives. But remember this – energy doesn’t always equal intelligence, so seek out the big thinkers over the big talkers. They’re just as likely to be introverts, only netter equipped with the intellectual persistence and the emotional smarts needed to help build a thriving culture.
1. Reward Wishful Thinking and Craziness – ever wondered where great business ideas from the likes of AirBnB came from? Probably from folks challenging the status quo by creating problems customers didn’t know they had (and then solving them with software)– like – what if we could take that wasteful spare room you have and turn it into a cash generating mini-hotel?
In a DevOps context, we can use a similar approach. Looking at the ways processes are being conducted and then suggesting counterintuitive alternatives. For example, what if our change management process is completely wrong – can we find ways improve automation for agile development? Why should we wait for production monitoring – what if we build it during coding and testing to increase resilience?

In a strong DevOps culture suggesting ideas is openly encouraged and rewarded. Without it, great ideas get lost or unnoticed, or worse, get belittled and smashed by those elitists and narcissists.
1. Have a Healthier Respect for Failure – if you want to kill DevOps before starting, go ahead and talk up past failures – like – “We tried agile five year ago and it was a train wreck –DevOps will be no different!” Glass half-empty thinking like this is detrimental because it put the brakes on talking risks and exploring a profitable digital future.
Nobody likes talking about failure, but the key to making progress with DevOps is to accept that mistakes happen all the time. What’s key is treating every fail or screw-up as a learning point. That way we stop talking about things that didn’t work before, and start describing them as lessons which help build a brighter future.

Taking the fear out of failure needs a lot of executive support. Management should begin a parallel process of rewarding sensible risk taking and removing any structures and incentives that stifle innovative thinking.
1. Walk the People and Process Pipeline – even with great tools, DevOps won’t deliver anything if sub-optimal processes or organizational friction persist. In most cases defects and delays can be put down to people enforcing rigid policies or relying on manual and error prone handoffs.
One good practice taken from the world of manufacturing is taking a “value walk” across your own software factory. This involves spending time interacting with people as they conduct their jobs (develop, test, release etc.) – not to point fault, but to understand any situational stresses that increase waste and technical debt, and could be remedied from automation.

Carving out opportunities from the digital chaos and disruption is the perfect place for DevOps. But don’t get carried away – change is hard. Make sure you have the right leadership, practitioners and counterintuitive thinking needed to start your transformation journey– and keeps everyone onboard.
November 2, 2015
Piecing Together the Elements of DevOps Transformation
Sometimes when a concept as transformative as DevOps comes along, it’s easy to forget that subsequent adoption represents a highly complex and iterative process.

While it’s clear that some organizations are already deeply leveraging DevOps to embrace the Applications Economy and Digital Transformation, it’s no surprise that for many organizations this process has only just begun.

So where have today’s organizations made the most progress, and what are the most significant challenges that practitioners continue to face?

To get a better fix on the current state of DevOps maturity and isolate the specific areas where most organizations must continue to focus their attention, researchers at Freeform Dynamics have created a matrix of practices key to measuring progress, dubbed the “DevOps Jigsaw”.

Drawn from the same survey data recently surfaced to profile leading-edge “Digital Disruptors” that have already wrapped their arms around DevOps advancements, this latest report, sponsored by CA Technologies, offers a crucial view into the current environment.

It’s worth noting that of the 1000-plus respondents included in this particular data slice, the vast majority, or 80%, are categorized as “Early or Limited” adopters, with only 20% recognized as “Advanced”.

Of the nine individual DevOps “puzzle pieces” examined in the report, experts grouped these efforts under three umbrella categories: “Key Enablers and Controls”, “Skilled and Collaborative IT”, and “A Business Led Approach”.

Business Led Approach – This subset encompasses strategy and objectives required to tie DevOps adoption to larger business goals, including:
- Well-defined strategy and objectives – In this key planning stage of DevOps transformation, 55% of respondents said that much of their work is “already done”, while another 36% charted it as “a work in progress”.
- Business stakeholder education – Roughly 33% of respondents said that this informative step in DevOps adoption is complete, with a larger majority of 43% noting that they are still working to advance this process.
- IT-business alignment of priorities – In getting stakeholders on the same page around business and DevOps direction, some 37% indicated that they are nearly done, with 43% of still attempting to embrace these efforts.
Skilled and Collaborative IT – This genre of requirements covers the cultural and automation capabilities necessary to foster DevOps success, including:
- Relevant IT knowledge and skills – Related to the development of DevOps expertise, 39% of respondents feel that they’ve already made the grade, with 40% conceding that they have more homework to complete.
- Cross-functional IT processes – As much as any related initiative, DevOps mandates tighter internal collaboration, with only 35% of respondents having achieved this goal, and 45% citing the need to keep pushing.
- Cultural harmony within IT – On a related note, DevOps teams must work together smoothly to succeed, with a mere 29% of respondents claiming such parity, offset by 46% that acknowledge a teamwork deficit.
Key Enablers and Controls – This category of DevOps maturity addresses key issues of technology, solutions acquisition and IT security, including:
- Right infrastructure and tooling – Efficient tooling and infrastructure are another key DevOps enabler, with some 37% of respondents feeling good about their current solutions, and 42% still looking for answers.
- Right suppliers and support – In sourcing the products and services required to advance DevOps process, only 33% of respondents said that they’ve found the right providers, with 46% still seeking additional help.
- Security and compliance measures – No area of IT can be addressed without consideration of security and audit issues, with 35% of respondents comfortable with their controls, and 46% in need of added defenses.
Given these overwhelming results, it’s clear that most organizations still have a long way to go in achieving greater DevOps maturity; at the same time, there’s also plenty of evidence that today’s practitioners have already made a good deal of progress.

Comparing your efforts to these established “DevOps jigsaw” puzzle metrics should help inform future strategy in creating a more cohesive framework of people, process and technologies, and deliver DevOps success.
October 29, 2015
DevOps Must Also Mean OpsDev

Why DevOps Communications Need to Be a Two-Way Street

Today, mobile apps have become one of the most important components of brand awareness and opinion. Often considered a company’s “front door,” a mobile app is an important way for consumers to directly interact with a brand, so it’s key for organizations to deliver high-performing, feature-rich user experiences right from the start. The margin for error is low – according to industry statistics, 79 percent of end users will only retry an app once or twice if it fails to work the first time, and only 16 percent report they will give it more than two attempts.

With these requirements in mind, the role of an organization’s development team is more important than ever. In today’s hypercompetitive market, organizations must get applications to market as soon as possible, without compromising functionality and end-user performance. This has led to the emergence of DevOps, a new approach to software development emphasizing greater collaboration, agility and speed across the entire software development continuum – developers, testers and IT ops teams – in order to meet the joint goals of velocity and quality.

An interesting by-product of the DevOps movement is the blurring and blending of roles across the application lifecycle as a result of these condensed timeframes and pressure to deliver new functionality, particularly in the area of testing and quality. This can be a boon, however, as we’ll discuss.

“Shift Left”: Eliminating Defects as Early As Possible

The traditional application lifecycle followed this model – ideation, development, testing and finally, production. It used to be that testers assumed the bulk of responsibility for ensuring an application worked as it was supposed to, before it moved to the production phase. But today, developers are assuming a greater role in testing, ensuring that applications meet basic functional requirements before they move any further down the funnel towards production.

This is known as the “shift left” movement, and ultimately it helps prevent code defects from getting baked into applications as they progress through the lifecycle. The further along an application moves toward production, the more “baked in” defects can become, and the more costly and time-consuming they are to un-do.

Even with their newly added responsibilities, the world of agile demands developers to work within shortened testing cycles. It is a race against time. Fortunately, tools exist today to bring greater testing automation to developers, plugging right into existing developer environments and frameworks. Further downstream, automation tools for testers enable them quickly provision (and de-provision) resources, and tweak testing environments. By reducing sluggish, error-prone manual processes, organizations can better deliver velocity and quality simultaneously.

“Shift Left” Also Requires A “Shift Right”

Ideally, an organization doesn’t want to treat real end users as beta users. But as the saying goes in software development, “the last bug is a mirage.” The reality is, even with all the due diligence on the left, finding some bugs in production is normal, and the “zero bugs” goal – while sounding commendable – can actually kill motivation, hurt creativity and discourage healthy risk-taking. Many developers note they spend much more of their time modifying existing lines of code, than writing new code. In a sense, testing – and the pursuit of better and better applications – never ends.

This is when IT ops teams can start making own contribution to application quality – a “shift right” so to speak. There is a wealth of information in production data that can be fed back to developers, in order to further optimize application functionality and end-user performance. Forward-looking IT ops teams are going a step further by applying data analytics, yielding deeper insights that can then be fed back to development teams.

Consider the case of SpeechTrans, a leading speech translation company. SpeechTrans applies Big Data analytics to end-user experience data – for example, what screens are end-users getting stuck on the most; where are end users exiting the app – so developers can focus specifically on improving these app areas. Previously, it took SpeechTrans up to three weeks to fix an app issue; now, they can address most problems in less than two days. This approach has enabled SpeechTrans to achieve consistent 5-star ratings in mobile app stores, a key driver of new customers and revenues.

Testing will always be indispensable in the development process, but further value can be realized by releasing an app into the wild and collecting real end-user data. IT operations teams can help developers and testers optimize applications in other ways. For example, they can support Canary Releases (or “rolling roll-outs”), enabling developers to roll out an application to a finite number of end users, in advance of a wider release. In addition, another important component of high-quality applications is cost- and resource-efficiency. IT ops teams can share production data on application usage patterns (such as, when during the day does an application come under heavy load, and when does this load taper off). This enables development teams to code applications accordingly – for example, architecting applications to accommodate longer or shorter queues (depending on time of day) which can free up CPU resources.

So in this way, when “OpsDev” happens, IT ops teams are helping developers do their jobs, but developers are also architecting applications with IT ops in mind, helping them utilize resources better. The days of developers simply creating applications and “throwing them over the fence” to testers and IT ops teams are over. Additionally, with real-time insights into production-level data, IT ops doesn’t just point a finger at development when a code-level problem arises. Rather, IT ops can provide deep, data-based insights into problems, giving developers a chance to prioritize and proactively address fixes. Advances in Big Data analytics are an important part of this.

Conclusion: Application Quality is a Shared Responsibility

To be truly successful in the development and testing process, organizations are recognizing that application quality needs to be addressed as early as possible, to prevent as many defects as possible from getting further embedded and ultimately reaching production. But like many aspects of the continuous delivery model, ensuring application quality requires a continuous feedback loop across developers, testers and IT operations teams.

In a continuous agile delivery environment, no one can shift responsibility for application quality to one group or another. The entire DevOps value chain – developers, testers and IT ops teams – must be involved and committed to collaborating as closely and efficiently as possible. In the true spirit of DevOps, lines are being blurred, between not just developers and testers, but also developers and IT ops teams. Silos are being transcended and “left-to-right,” back-and-forth communications streams are taking precedence. When “DevOps” also becomes “OpsDev,” the traditional, linear approach to development is replaced by a model that better supports the common goal – bringing high-quality software and enhancements to market, quickly and efficiently./

About the Author/Eran Schitzer

Eran Schitzer is a director of product management in the Application Delivery Management business unit within HP Software, responsible for driving the direction and release of new and existing products within the Performance Testing and Lifecycle Virtualization product suite.

He has more than 15 years of experience launching cloud, mobile and e-commerce products for consumers and enterprises, at companies ranging from start-ups to large-scale public entities.

Since joining the company, Schtizer has led the acquisition of Shunra Software, a network virtualization company; defined and executed the cloud testing agenda, and driven the move from an on-premise to Software-as-a-Service consumption model, including the launch of StormRunner Load.

Schitzer holds bachelor’s degrees in Economics and Management as well as Information Systems from Technion – Israel Institute of Technology. He also has a master’s degree in Business Administration from the University of California, Berkeley, Haas School of Business

October 27, 2015
Does Agile need continuous integration?

Michael Oubre – Agile evangelist, practitioner and trainer, and Marc Hornbeek – DevOps continuous testing evangelist and thought leader, share their views on popular questions regarding Agile and DevOps from their respective viewpoints.

If you are doing Agile do you need to do Continuous Integration?

Michael: From an Agile perspective, let’s consider your company decided to take the plunge and switch its software development process to Agile, formed Agile teams, got them some training and set them up with a coach. The team have been scrumming for a couple of iterations but there’s not a lot of work getting finished, and many of the stories are having to be carried over from sprint to sprint to get the defects resolved.

You might be tempted to start worrying about the quality of the code being written, or whether you have the right ratio of test engineers, but it might be more useful to look at your build/test cycle. One of the big benefits of Agile is the reduction of the time between the code being written and the identification of defects – the shorter that time is, the lower the cost is to get it fixed. Agile removes the process barriers, but if you only rely on unit testing and infrequent integration testing, you won’t achieve the results that you hope for.

Agile teams will have a very difficult time of ever reaching high performance without the benefits of Continuous Integration providing the ability to deliver new code in an integrated, testable build to the Agile teams as close to “instantly” as you can manage.

Marc: From a DevOps perspective development teams that are “doing Agile” are the primary customers of Continuous Integration together with Continuous Testing.

Both Agile and DevOps emphasize the importance of incremental development processes rather than large, serial waterfall processes because incremental processes facilitate fast feedback and reduce process waste compared to long waiting periods between software creation and validation. In fact the DevOps “movement” was born out of a system test consultant’s frustration with the disconnect between some Agile development teams and separate operations teams. (Reference: http://itrevolution.com/the-history-of-devops/) . Today the words Continuous Integration and DevOps, together with Continuous Testing and Continuous Delivery are practically synonymous.
The mere fact that there are plenty of successful Agile examples that existed prior to DevOps or Continuous Integration became a word tells you that Agile can indeed be done without CI, but clearly CI is a great complement to Agile and supports the goals of Agile while helping to tie it to operations. So the strict answer the question “If you are doing Agile do you need to do Continuous Integration?” is, well, “no” but to leave it at that would be incomplete, so a better answer is “ no, but CI is definitely recommended to complement Agile”.

Bottom line is, from both Agile and DevOps perspectives continuous integration is really important to achieving performance goals. At Spirent we think Agile and DevOps work together to help teams develop and deliver quality software efficiently. More information about Spirent’s views can be found at the following link: Spirent.com/solutions/devops

What do you think of these views and do you have other views?

About Michael Oubre

Michael Oubre is an expert in Agile practice, transformation and management and has spent more than 10 years as a practitioner, coach, trainer and evangelist. His long and eventful tech career has given him practical experience and insight into every aspect of building, deploying, supporting and managing software, systems and networks.

October 26, 2015
SAFe and DevOps – a recipe for a delivery smoothie!

DevOps brings you the deep pink color of fresh strawberries and SAFe the rich, creamy texture of blueberry yoghurt!

‘Agile’ delivery is not always smooth. If not managed well, it can be a nightmare, as those of us who have seen rogue agile processes run amok know. That’s where the Scaled Agile Framework (SAFe) comes in. It blends with DevOps to bring agile delivery to the enterprise.

A quick word about SAFe: Those familiar with SAFe would know that it divides the work of building a system into small time-boxed units. Each unit must typically be deliverable in around two weeks.

For instance, if you run a factory then receiving and issuing material is likely to be part of what you want to automate. You may want to provide users with features, such as the ability to create issue vouchers online, modify vouchers, cancel vouchers, or search for a particular item while creating a voucher. Some of these features may be small enough to be delivered in two weeks. Others may need to be split up further into sub-features, and so on. Let’s suppose the item search feature is tiny enough to be implemented in a couple of weeks and can therefore be considered a unit (or ‘iteration’ in SAFe parlance). The system would be the sum of several similar units, each capable of being implemented in a couple of weeks. Typically, one would assign the units to various teams, and schedule them based on their inter-dependencies and various staff, infrastructure, and other resource constraints. This composite set of small units is called a ‘program’ in SAFe. More about SAFe in a later article.

The DevOps culture: The SAFe methodology requires that each unit be made usable within the two week interval. That is, the unit must be developed and deployed to a production-like environment. The ‘agile team’ creating the unit must therefore have an ‘end-to-end’ perspective of the unit and not be constrained by artificial boundaries of development, testing, or operations. The members should have the skills, and be willing to work in each other’s domains, collaborate actively, communicate frequently, and provide feedback quickly and clearly.

This active collaboration between developers, testers, and operations is what DevOps is about. An organization that has a DevOps culture will therefore find it easier to implement SAFe.

Developing on cadence: A factor which makes project managers reluctant to use agile methodologies in large enterprise scale projects is the number of multiple units (and teams) operating in parallel. SAFe attempts to simplify the process of synchronization between teams by aligning the two-week windows so that they all start and end together. By ‘developing on cadence’, as this practice is termed in SAFe, the different agile teams can align common activities like planning, review and system demos. This makes it easier for the program to identify inter-team dependencies and assess how unit-level feedback and change requests can affect other units or teams. The cadence mitigates the complexity of synchronizing work across scores of teams.

SAFe DevOps tools: While work synchronization helps, there are certain activities that still need tools. With often several hundred code changes being checked in every day, enterprise scale projects need sophisticated configuration management and deployment tools capable of tracking and maintaining the multiple versions of code, intelligently scheduling and triggering builds and automated tests, communicating feedback about test status, configuring and creating environments, creating installers, and deploying the builds to the appropriate environments. Tools like Docker make the deployment process quicker and more flexible. IBM® Rational® solution for Collaborative Lifecycle Management (CLM) provides a suite of tools capable of working across the spectrum of tasks and managing the complex delivery process end-to-end.

Agility over the last mile: Because of the need to deliver usable code at short intervals, testers and operations personnel need to be as nimble as developers. While developers may have achieved success in using Agile methodologies and have efficiently been churning out byte sized chunks of code at short intervals for a while, it is only more recently that operations teams have begun to accept agility as an achievable goal. DevOps, assisted by various collaborative and automation tools like CLM and Docker, has played a significant role in changing mind sets and transforming capabilities over the last mile.

It is this end-to-end blend of the DevOps culture with SAFe, along with the various tools that are part of the DevOps ecosystem that will propel enterprise level software delivery towards the goal of smooth, continuous delivery.

About the Author / Vidhya V Kumar

Vidhya is an advisory software engineer at IBM. An evangelist and practitioner of agile methodologies, continuous delivery, and documentation, she has around ten years of experience in the IT industry. She is a subject matter expert in DevOps, Application Lifecycle Management, and Search methodologies.

October 21, 2015
Microservices, Hyper-Complexity, and Automation
Imagine building a human being one DNA codon at a time. Sure, you might be able to build a pretty awesome individual. But it would take a long, long time. And you’d only be able to build one. Worse yet, if you didn’t get your person exactly right on your first try, it would take you another long, long time to come up with Version Two.

Unfortunately, this is exactly the position many companies are putting themselves in with microservices.

Yes, microservices theoretically enable companies to adaptively recombine their digital DNA to build the increasingly awesome apps. But without better automation, they won’t be able to do so very quickly—or very reliably.

Hyper-Complex DevOps

Microservices introduce significant additional complexities into the DevOps lifecycle. Each microservice has its own data mappings, APIs, latency/performance dependencies, etc. So, while microservices may offer genuine advantages in terms of adaptability and re-use, they also multiply the total number of discrete DevOps tasks—especially vis-à-vis test/QA—by at least one full order of magnitude.

This is a serious problem for shops that aren’t sufficiently well-automated, because any task—or, as is much more often the case, any handoff between tasks—that is not well-automated will quickly turn into a bottleneck.

The resulting adverse consequences of insufficiently automated microservices DevOps can thus include slow delivery, excessive cost, and mistakes. Worse yet, chronic bottlenecking will likely create a pervasive organizational disincentive that fundamentally undermines the microservices initiative itself.

What Automation Entails

Thorough, effective DevOps automation is, of course, more easily aspired to than achieved. To implement the kind of end-to-end DevOps automation required for microservices success, organizations have to get three things right:
- Process mapping. Effective automation requires an exhaustive definition of process. This exercise is not for the faint of heart—because most organizations have many pockets of ad hoc process at various points in the DevOps cycle. Test/QA tends to be especially variable based on factors such as the delivery pressure teams are under at any given time.
- Automation technology. Development teams often use scripting and other technologies to automate various DevOps tasks. An overly fragmented automation environment, however, can be difficult to orchestrate and manage. So the move to microservices may necessitate a more holistic integrated automation environment.
- Leadership and culture. A well-automated DevOps process alone is insufficient for microservices success, because it is unfortunately not true that “if you build it, they will come.” On the contrary, a strong leadership commitment is essential for changing the way people work.
The application economy is putting development organizations under intense pressure to be more productive and responsive to ever-changing market dynamics. Microservices offer one possible way to respond to that pressure. But microservices won’t work without a much more rigorous approach to DevOps automation. IT leaders may therefore want to focus on automation first and microservices second, instead of the other way around.

Unless, that is, you only want to build one app using microservices—and then maybe build a slightly better one a few months later.
October 19, 2015
Coming to DevOps Enterprise Summit 2015? Must attend IBM sessions and Night at the Garage event!
DevOps Enterprise Summit 2015 is next week. Though only in it’s second year, it is a conference that any technical professional can get excited about, and is an event that IBM is proud to sponsor. Three days of discussion span topics from product integration, to agile and lean concepts, and how to start implementing DevOps and continuous delivery at enterprise scale.

We are bringing the full force of IBM DevOps to #DOES15 this year, with subject matter experts speaking in two keynotes and two sessions, and bringing even more knowledge to booth #8 to show IBM DevOps solutions in action. If you are going to be at DevOps Enterprise Summit this year, make sure you check out:
- DevOps and Innovation at USAA and IBM from Dibbe Edwards, Vice President Development, DevOps for Hybrid, IBM and Michael Bueche, Assistant Vice President IT Operational Excellence, USAA on Monday morning from 11am-12pm in the Grand Ballroom. Dibbe and Michael will discuss how DevOps initiatives combined with automation and culture changes are driving quicker user feedback and overall software delivery.
- Act like a Start-Up, Scale for the Enterprise from Rachel Reinitz, IBM Distinguished Engineer and CTO, IBM Bluemix Garage, Dr. Angel Diaz, VP of Cloud Technology & Architecture, IBM, and Kristal York, Executive Mobile Strategist, PointSource on Tuesday afternoon from 2:15-3:15pm in Bayview A. Rachel, Angel and Kristal will discuss why and how IBM is implementing DevOps and how IBM’s global network of Bluemix Garages is providing the knowledge and experience that enables our clients to transform into digital businesses and deliver disruptive innovation.
- Test Automation For Mainframe Applications from Rosalind Radcliffe, IBM Distinguished Engineer & Chief Architect for DevOps and CLM on Wednesday morning from 9-10am in the Grand Ballroom. Rosalind will be talking about how to apply shift left testing to mainframe applications, including integration testing and virtual services to improve early testing.
- Organization Design Panel Discussion with Mustafa Kapadia, IBM DevOps Service Line Leader, as the facilitator join Jason Cox, Director, Systems Engineering at The Walt Disney Company, Jeff Einhorn, Senior Group Engineering Manager at Target, Jeff Raffo, Director – Personalization Technology at Nordstrom and Jim Penny, Solutions Director at IBM on Tuesday afternoon at 2:15 pm in the Exhibit Hall. The group will discuss about Organization Redesign and how some of the largest enterprises have been able to achieve this transformation. You may also get a box set of our For Dummies books or a limited edition IBM DevOps t-shirt with your visit!
Don’t miss out the Night at the Garage!

Finally, if you are looking for an immersive DevOps experience that includes complimentary beer, wine, local cuisine, and live entertainment, join us for a Night at the Garage! IBM is providing transportation on Monday night, October 19, from the Hyatt Regency SF to the Galvanize building that hosts the IBM Bluemix Garage, where you can network with other DevOps enthusiasts and witness the art of the possible firsthand at 5 interactive workstations. Register now to attend, space is limited! We look forward to meeting you there.

About the Author/Saleem Padani

Saleem Padani has 20-plus years of technology and business experience in product and application development, enterprise architecture, service delivery and solution marketing. Currently, he is responsible for overall go-to-market strategy and execution for IBM DevOps and Enterprise modernization capabilities for IBM. Follow him on twitter at @spadani1
October 15, 2015
How Fast Is Fast Enough?
One of the primary goal of DevOps is the ability of an organization to bring an idea to market faster. But how fast is fast enough? Do they need to improve time to market by 10%, 20%, 50%..90%? Do all applications need to ultimately be released within hours? In order to compete do we all have to be the Etsy or the Netflix’s of the world?

Unfortunately there is no simple answer. Optimal application delivery speed (aka time to market) varies drastically across industries, markets, organizations, and even within an organization’s own application portfolio. Remember “two speed IT”.

In other words there is no “one fit for all” when it comes to speed. Yet defining your optimal speed – goal – is critical to your DevOps success. Without it, its almost impossible to provide a vision, measure progress, and fundamentally change the way you delivery software. For example, IT is very good at measuring costs. In fact they measure costs from every way to Sunday. As a result, cost takeout becomes the fundamental value metric for service delivery – “how low can you go to delivery the same service”. Similarly, if speed and agility is the goal then we must start with measuring our current velocity and then setting (realistic) goals for the future state.

The Definition of Speed

Before we go about trying to determine an applications optimal speed it is important that we first agree on the definition of speed. We like to think that speed is a fairly simple concept, but its amazing how mis understood it is.

Many organizations new to DevOps simply equate speed to release cycles. They believe that if they increase the number of cycles then they are improving their time to market. Unfortunately it does not work that way. Improvement in release frequency is a natural outcome of going faster (as we will see in the example below) but not the driver of speed.

True cycle time is measured from end to end – when the idea enters the development funnel to the time it is released to the end user. In other words it’s the Work In Progress (WIP) time for that particular piece of code and any improvement in speed is the reduction of this WIP time.

For example, a healthcare client recently shared with us that it takes them just 20 days to develop a medium size enhancement. For them, their cycle time was 20 days per enhancement. But on further inspection we found that it takes 20 days to determine requirements, 10 days for setting up environments (while using cloud), another 15 days for testing, 10 days for moving code from one environment to another, and a whopping 15 days for approvals. When we added all that up, we discovered that their true time to market (speed) for that enhancement was just over 90 days.

Once we were able to define speed in tsurehis context, it opened up an entire new world of dialogue within the organization. Now instead of optimizing 20 days, they have another 70 days of activities to pick on. Even if they were to shave just a mere 1/3 of the 70 days (cue automation with some simple processes engineering) they would be able to improve their time to market to 65 days. And as an added bonus, the team instead of just delivering 4 enhancements per year (90 days x 4) they can now deliver 5 enhancements (65 days x 5) much faster.

By defining speed in such broad terms, it not only gives IT teams a glimpse at the big picture but also visibility into areas of wastes, opportunities for true end to end improvement, and a platform for collaboration.

Now that we have our definition right, lets get on to answering the million dollar question,

How fast should I be (what is my optimal speed)?

The simple answer is fast enough to delight the customer and beat the competition. But to determine at what point you are able to achieve this goal is a bit more complicated.

Most organizations try to answer this question by looking inward. They perform a detailed value stream mapping exercise, determine cycle time for each activity, identify bottlenecks, and then triangulate how fast they can go if they eliminate some of the key bottleneck.

Unfortunately, speed is not a function of how fast can you go (IT view) but a function of potential benefits (business / market view). More precisely, your optimal speed is directly depended on the additional benefits the business will enjoy (additional revenue, higher customer acquisition, better customer retention etc.) as a result of greater velocity.

To help understand this relationship between benefit and speed better, ask yourself the following three questions,
1. If we were to reduce our cycle time (from idea to production) by 10%, 20%, 30%, 40% etc., what is the incremental benefit to the business
2. Can we quantify this benefit?
3. How sensitive is this benefit to speed? Are they inversely proportionate, is there a benefit / speed plateau?
By doing so, there is a good chance that most applications will fall within one of the three scenarios.
1. Benefit and Speed Are Directly Proportional
For some businesses, speed is king. The faster you can deliver a product the greater the incremental benefit. Consumer facing apps (Facebook, Linked In etc.) focused on revenue generation / customer acquisition work well in this scenario. For these applications, there is essentially no speed limit. The rule of thumb is step on the gas and get out of my way.

Optimal Speed: No speed limit, step on the gas
1. Benefit Improves But Only To A Certain Point
Most SaaS applications (SFDC, Workday etc.) fall within this category. At a certain point the incremental benefit diminishes as speed increases. In other words, users like the regular updates and upgrades but to a certain point. Change too much too fast and you end up delivering little additional value. In this case, there is an optimal speed and any investment in speed after that is a waste of money.

Optimal Speed: Limit your speed to the point where benefits plateau
1. Benefit and Speed Are Adversely / Not Related
This is the opposite of A, where speed has an adverse / no impact on benefit. Here users prize stability and status quo over anything else. Changes are welcomed but need to be few and far between. Many on-prem legacy applications fall into this category. Imagine if your ERP provider published a new update (mini upgrade) every day. It would wreak havoc to the organization. If this happened to your general ledger system, you would never be able to close your books. If you find yourself in this category it is best to go pick a different application.

Optimal Speed: None, stability is prized over changes

Two Caveats

The graphs are a great representation of the various options but there are two caveats that every organization should be aware of, cost and competition.

Cost: The speed limit makes little sense if the cost of getting to that speed is greater than the incremental benefit potential. After all, like any other IT project ROI is key in deciding if DevOps is worth investing in.

Competition: Ultimately…competition holds the trump card. No matter what your charts say and / or your cost model shows, if the competition is going faster…you have no choice but to match that speed or better yet, beat that speed. Which is why we urge many of our clients to invest into DevOps early, set the benchmark, and build a competitive advantage.

Happy speeding!!

About the Author/Mustafa Kapadia

Mustafa Kapadia is Service Line Leader for IBM’s DevOps practice, a business advisory practice focused on helping large enterprises transform their software & application delivery. Through DevOps, organizations can get software to market faster, with better quality, less defects, reduced risk and shortened lead times.

Prior to joining IBM, Mustafa was a management consultant with Deloitte’s Strategy & Operations practice. He lives in San Francisco Bay Area, is an avid blogger (when time permitting), a speaker, and and advisor to start ups.

You can follow Mustafa on LinkedIn: https://www.linkedin.com/in/kapadiamustafa and Twitter: https://twitter.com/mkapadiatweets
October 14, 2015
Native Integration of APM Driving DevOps Maturity

With applications performance monitoring established as one of the fundamental cornerstones of today’s DevOps methodologies, it’s easy to overlook the reality that effective instrumentation of such tools remains extremely difficult. Specifically, the biggest challenge remains creating performance metrics that address both IT and larger business initiatives.

There’s no question that APM is one of the most widely adopted and deeply embedded elements in today’s DevOps world, serving as the critical source of measurement that enables more transparent and agile applications. Yet, despite the fact that performance monitoring solutions, and their respective providers, are considered to be among the most mature on the DevOps landscape, customers still struggle to optimize the systems to meet their specific requirements.

However, as the market continues to advance, specifically via direct integration of widely-adopted APM tools with other strategic applications management systems, such as e-commerce platforms, organizations are finding a faster path to success, and bigger-picture DevOps benefits. One such partnership, announced in late August, saw market leader New Relic expanding integration with e-commerce platform provider Magento, owned by eBay.

Jon Azrielant, director of marketing at online retailer Jewelry.com, a Magento customer, said that his company initially ran into significant obstacles in adapting New Relic’s APM capabilities to its processes, despite the comparative strength of the tools.

“We had a really difficult time implementing New Relic initially, it was a painful process,” Azrielant said.

While the APM tools provided Jewelry.com with the data necessary to generate vital alerts, along with more agile sysadmin and operations monitoring, the more silo-based, fragmented approach that it employed could not empower the retailer to circumvent traditional walls between apps support and service support teams, Azrielant said.

Leveraging the extension of New Relic’s Software Analytics Cloud within Magento, the company has created more advanced instrumentation to inform critical business decisions. Among the most helpful of these measurements are those relating directly to customer experiences, while correlation of such metrics with Jewelry.com’s bottom line has also advanced executive decision-making, he said.

“The critical result of this integration is more complete visibility across applications and business performance, importantly tied to the actual business ramifications,” said Doug McIver, director of product for Magento. “In the past, merchants had to do a lot of difficult integration to use these systems together, so obviously this is a hugely valuable capability for us to be able to offer, especially to customers with highly performant site requirements.”

Moreover, Magento claims that it is growing its own customer base by tying APM more closely to its platform. Likewise, New Relic officials concede that to help advance crucial DevOps workflows, it must partner with such platforms to deliver APM data within the context of business requirements.

“If we’re talking about how to win in world where we’re iterating the end users’ experience on a regular basis, we have to put performance metrics within context of the UI, and the business results; working from the same data sets and getting people on the same page is incredibly important,” said Abner Germanow, senior director of solutions marketing at New Relic.

While pure performance stats are “near and dear” to New Relic for obvious reasons, the solutions provider is also attempting to advance a broader array of information that drives closer collaboration through new “iteration and experimentation”. New Relic is also focused on further developing the specific types of information most critical to key constituents such as e-commerce platform users. The need to create a common language for those DevOps efforts is amplified by the fact that development and operations officials are often unsure which metrics are critical to the business, and vice versa.

“Instrumenting code has always been a challenge, and we’ve worked very hard to ease that pain across a lot of different platforms; there’s always another language where work can be done, and we look to our customers to help lead us into the next phase of instrumentation,” Germanow said. “We’re seeing a lot of pressure around the need for constant change and iteration in industries where it’s truly a matter of survival, such as e-commerce, publishing, digital governance and finance – places where the industry is behaving very differently now than it has over the last 50 years.”

This work, and the subsequent APM integrations offered via platform providers such as Magento, have allowed end user organizations to remain focused on customer experience and sales, the ultimate goal of DevOps, said Jewelry.com’s Azrielant.

“In terms of how this relates to the DevOps revolution, this is the advancement that is bringing everything together and allowing everyone to look at the same numbers, to share common APIs, to streamline reporting and event response, and define everything in uniform terms,” Azrielant said. “Ostensibly, the most basic tenet of DevOps centers around the unification of these disciplines, and you can’t support that without the unification of this information.”

September 29, 2015