devopsy.kinsta.cloud

Tag: agile

Librato Seeks to Unlock DevOps Application Monitoring

As wider adoption of DevOps methodologies continues to foster demand for new tooling to help streamline evolving workflows, at least one vendor is attempting to address the growing need for related monitoring capabilities.

Librato, a San Francisco-based provider of Amazon AWS applications performance monitoring (APM) services, claims that it is finding favor among the rapidly expanding DevOps community driven by its ability to inject critical visibility into the engineering lifecycle.

Acquired by Austin, Texas-based SolarWinds in January for a lofty $40 million and renamed Librato SolarWinds Cloud, the company is touting its recent adoption by well-known providers such as Slack for monitoring of DevOps-centric applications performance and health metrics.

“At its core, DevOps is destroying conventional wisdom around ritualized processes and the need to perform them, mostly in the domain of scale,” said Dave Josephsen, Developer Evangelist at Librato. “By attempting to include monitoring as a key element of the development lifecycle, this has created a whole new range of challenges, and opportunities.”

By providing a centralized Web interface and open API to help create visibility into critical applications engineering requirements, Librato maintains that it is creating the tooling necessary to empower integrated development and deployment processes, such as continuous change control.

“There are a lot of disparate tools for monitoring individual engineering tasks, but there hasn’t been any practical way to bubble up that data for efficient monitoring,” Josephsen said. “Within the context of DevOps, change control is a ritual that we’ve abandoned as we’ve embraced test driven development and automated away the human element, so there’s a distinct need for monitoring of feedback and metrics.”

For example, cloud-based productivity messaging specialists Slack are using the Librato “StatsD” lightweight daemon process to stream business and app-health metrics regarding PHP applications into the monitoring platform. Slack also leverages Librato to chart data regarding connect, query and fetch latencies for its server clusters, and to monitor internal metrics about overall system health.

“Having those graphs side-by-side lets us clearly see and compare the effects of network latency on one of our new services, and that’s very cool,” Richard Crowley, director of operations at Slack, wrote in a recent testimonial posted to the Librato Solarwinds blog.

Industry experts agree that broader adoption of DevOps strategies will continue to drive demand for relevant monitoring tools, with solutions such as Librato seeking to deliver “modern stack architecture for looking at the modern stack.”

“I think that APM is always interesting because it’s always new,” said James Governor, industry analyst with RedMonk. “The pace with which we’re building new architectures to deliver services to end users is increasing all the time, and with every new wave of technology, we tend to get new and associated models of APM tooling.”

Librato is taking advantage of the need for a system that brings together the elements necessary for cloud native monitoring, and in an agentless manner different from other competing tools such as New Relic, the analyst noted.

However, whether or not the platform is able to break down traditional engineering siloes to enable more efficient DevOps management, as Librato officials claim, remains to be seen, he said.

“Each wave of technology has promised to do this, to remove the inherent finger pointing, and each has always made the same claim to break down organizational firewalls,” Governor said. “DevOps hasn’t boiled the ocean in this manner yet, but there are some unique opportunities; unlike previous waves, where automation was intended to replace humans, this is now about augmenting people, and we are seeing that teams are choosing new tools from which developers can truly derive value.”

August 10, 2015
Scaling Lean and Agile for the enterprise: All hands on keyboard!!!
Lean and Agile methodologies are nothing new, but in the last few years we have seen the emergence of several frameworks combining elements of both. The IBM DevOps framework is one clear example. Combining principles of Lean and Agile into a single framework enables scaling of the core benefits to the entire organization, not just pockets of experts dispersed across functional silos. The goal?
- faster release cycles, increased responsiveness, reduction of waste, lower cost base
- increased software quality, customer-centric feedback loops, cross-discipline collaboration
That said, these process transformation initiatives too often remain confined within the Engineering or IT side of the house. And given that more and more large scale software projects (mobile and web) are driven and managed out of the Marketing or LOB organizations, we need to get to a place where Engineering/IT and the Business share a common vision and framework for continuous software delivery. This blog will discuss key areas where the Business needs to engage to truly scale these benefits to the organization level.

Let’s take a look at the key principles behind these frameworks and highlight those which mostly impact the behavior of analysts, project/program managers and LoB executives on the business side:
- Optimize the whole: Development teams cannot get it right by themselves. The business must be actively engaged to ensure customer/user needs are properly understood, in-turn ensuring the intended outcome of the application/system is well communicated and understood, and eventually the overall effort is continuously steering toward the right business outcomes
- Eliminate waste: Identifying/removing wait time to reduce overall turn around and cycle times (active participation during needs/requirements analysis, timely inputs into work prioritization, immediate feedback on early prototyping/release). Discover the hidden factory!!!
- Empower people: Let the real user directly represent their needs, and build interaction systems and processes that make it easy for those with the needs to engage across the lifecycle
The most common symptom that engineering and the business are not in sync is a constant flow of “management urgencies” that are forced into the current sprint or iteration. These “must do now” urgencies are a massive source of distraction for the teams executing the work on both the business and the engineering side. Directly responsible for substantial loss in productivity across the whole chain, as well as a prominent source of friction in relationships between business and IT, these distractions and interruptions can be avoided with the right level of orchestration and collaboration.

So which are the crucial touch-points where the right level of engagement from the business can generate the most impact?
- Early definition of business needs: This is not about producing lengthy system requirements documents, it’s about interaction with software architects and designers early and often to properly describe the needs/expectations of the target users/customers, it’s about making sure the overall context is defined and understood. There will be several opportunities to get into detailed requirements as the work unfolds.
- Early involvement into definition of success/acceptance criteria: When I was a BA, little humbled me as much as thinking I had done a good job describing a business requirement only to then freeze when I was trying to define objective criteria that would serve to develop user acceptance test plans. Forcing the definition of acceptance criteria at the time requirements are defined often surfaces areas of the needs that could be better defined or drives important exchanges between engineering and the business.
- Early and on-going engagement into prioritization: Gone are the days when setting up priorities was the result of a monthly or quarterly exercise; customer/user behaviors change continuously, so should your alignment to their needs. Prioritizing monthly or quarterly is like driving your car on the freeway in a pitch dark night and turning on your headlights every 2 mins or so to get an idea of where you heading… a lot can happen in 2 mins driving at 100km/hour. Prioritizing is also no longer an exercise in faith/best-guesses/loud voice in the room…. this new world is about constantly adjusting based on validated learning; running A/B testing, driving early prototyping and experimentation, monitoring application usage, social media analytics all serve to drive the right backlogs/priorities of teams across the software delivery lifecycle.
So where to start?

This transformation is a journey, not a one-time call to action, different paths can be taken depending on your organization’s goals, culture and priorities. We’ve helped hundreds of organizations over the years plan their way forward, helping them structure teams and investments while respecting their constraints and objectives. Be bold, take this on-line DevOps Practice Self-Assessment! Or visit our blog on IBM’s support for the Scaled Agile Framework (SAFe).

Also, do visit us at Agile 2015 to experience IBM DevOps solutions for collaborative development, continuous testing and continuous release and deployment.

About the Author/Roger LeBlanc

Roger is the Strategic Offering Lead, DevOps Application Platform driving the Enterprise Scaled Agile solution. He joined IBM following the acquisition of Systemcorp ALG, serving as Managing Director, Professional Services and has since been involved in ensuring a smooth transformational journey for the clients. He has a credible perspective and proven track record for meeting executive stakeholders expectations and goals. Roger has contributed to revenue growth and customer successes as well as provided strategic oversight and guidance of customer engagements.

Connect with Roger on
August 5, 2015
Tailoring DevOps to your IT: A case study for SAP
I am frequently asked if DevOps principles can be applied to legacy environments such as SAP. They clearly can, with some tailoring. But you should be aware of certain constraints.

An agile transformation from conventional IT (where the SAP software delivery process is split between business analysts, development, the QA department and IT operations) towards an agile DevOps organization requires the following steps:
1. Organize development teams by business capabilities rather than technology – when defining the structure, ensure teams can work as autonomous and independent as possible.
2. Upskill business analysts to product owners.
3. Organize development output by agile principles using SCRUM or KANBAN.
4. Successively move QA into development teams – a separate QA team can take responsibility for automation of regression, security and performance tests, but should not be involved into day-to-day testing activities.
5. Successively augment development teams with job scheduling know-how.
6. Introduce automation tools to ease the promotion (transport) process for changes into upper environments and, ultimately, production.
7. Add unit test and static code analysis (e.g. for security and performance checks).
8. Automate regression, security and performance tests.
9. Focus responsibilities of the SAP platform team on managing the platform (monitoring, patching, upgrading, environment provisioning and refresh, performance testing) and leave the transport of changes (“deployments”) to development teams.
10. Use automation tools for environment clone and refresh.
11. Move the SAP platform team towards KANBAN.
While these steps are similar to agile DevOps transformations in other technology contexts, the specific automation tool chain is very different.

Instead of using Git, Jenkins, Ansible or Chef, you are dealing with tools from SAP and third-party vendors for code inspection and analysis, transport management, test automation, environment refresh, performance testing and more. This requires skills that can be hard to find. In addition, different tools may be used to cover different SAP platforms (the “ABAP-Stack” and the “Java-Stack”).

There is another peculiarity for SAP: Although new technology supports fast cloning of isolated SAP environments and their continuous refresh with current data, this is constrained to certain infrastructure. The costs and lead times to provision and manage a single SAP environment remain relatively high.

Therefore, several teams will often have to share a single SAP environment for development, testing or release purposes. This requires coordination and sets limits to continuous delivery. In the best case, daily production deployments can be achieved by opening the pre-production environment for transport and application tests over a couple of hours, then locking it for another couple of hours for regression, performance and security testing (including batch) and production delivery.

In summary, many principles of DevOps can be applied to SAP application delivery, if peculiarities are taken into account.

While continuous deployment for SAP environments remains an ambition, daily releases can be achieved today.

DevOps Dozen Awards! Automic is currently nominated for a DevOps Dozen Award in the Release Management and Continuous Delivery categories. Vote for us today!
August 4, 2015
How ADLM Gobbles Up DevOps

The Next Evolution of Software Delivery Automation

In the 1980’s and 90’s, the business software landscape was dominated by a diverse list of cutting-edge companies such as Best Software, i2, Brock Control Systems, Mapics, Ross Systems, Infinium, FBO Systems, Manugistics and MSA (of course I could go on and on). Now long gone, these and hundreds more really great companies have been gobbled up or rendered obsolete by the rising class of ERP giants. In this article I’ll explain why history will repeat itself leading to the extinction of most DevOps tools as leading ADLM platforms continue to assert their dominance across the diverse software development and delivery automation ecosystem.

History Informs Our Future and The Evolution of ERP

You already know that for the past twenty years, most companies have leveraged some form of ERP system to manage virtually every core business processes. One benefit of this tightly integrated solution is a powerful inter-functional data flow that enables corporate agility and provides the highest level of visibility. This super-integrated architectural model has become the standard adopted by virtually every enterprise around the globe. What you may not know is today’s “ERP model” is the result of four distinct evolutionary generations that I believe help predict the next major evolution of automated software delivery.

Phase 1 – Automation
Enterprise Information Systems (EIS) – In the 1960’s, early automation systems were developed to support important individual business functions such as general ledger, inventory management, billing, payroll, etc. These systems were architected completely independently of each other and added little value to the enterprise beyond their narrow scope.

Phase 2 – Core Data Model
Manufacturing Resource Planning (MRP) – Then in the 1970’s, the idea of a master production schedule was devised so that a few of these isolated systems could gain greater visibility into future inventory and product requirements of the organization. The big idea behind the master production schedule was the creation of an open data model that could be leveraged by other systems impacted by the manufacturing production schedule.

Phase 3 – Expansion
Manufacturing Resource Planning II (MRP II) – In the 1980’s, software vendors began to build and sell off-the-shelf packages that promised “best of breed” process design. These solutions provided tightly integrated versions of the key manufacturing processes required to produce products.

Phase 4 – Business Process Domination
Enterprise Resource Planning (ERP) – Finally, in the 1990’s, business software vendors expanded well beyond the manufacturing scope by creating highly-integrated solutions that now cover just about every standard business function imaginable. These systems leverage a unified data model to dramatically improve visibility, consistency, accuracy and planning capabilities enterprise-wide.

What Does ERP Have To Do With Automated Software Delivery?

The evolution of ERP has taught us how natural pressures forced the creation of a unified and comprehensive “business data model” spanning the entire enterprise. Those software vendors with enough influence to dictate that data model were the ultimate winners in the ERP space.

In the very first generation of ERP (EIS), software was leveraged to deliver a high degree of automation to many business processes that were previously manual. Over time, it became clear that these newly automated processes were interconnected and the evolution towards a tightly integrated and unified data model was underway. The business objective that fueled each successive generation outlined above was the need to design more efficient business processes that increased organizational visibility and agility.

As Marc Andreessen famously said in 2011, “…more and more major businesses and industries are being run on software and delivered as online services—from movies to agriculture to national defense.” (Why Software Is Eating The World). That statement resonates even stronger four years later. Today, virtually every corporate organization is seeing the familiar pressure to deliver software more efficiently and reliably. If history is indeed our guide, any highly fragmented and/or isolated process required to deliver incremental software change will face mounting pressure to be merged into an integrated end-to-end enterprise-grade platform that can deliver improved cross-functional visibility with even greater efficiency.

Why Application Delivery Lifecycle Management Will Win

In my view, there are really only two broad solution categories in the realm of automated software design and delivery – Application Development Lifecycle Management (ADLM) and the catch-all term DevOps (which I’ve hijacked here to describe any other type of process automation that assists the software delivery process).

DevOps tools are often narrow point solutions that have sprouted from open source projects, in-house development or commercial vendors. Organizations rely heavily upon a diverse collection of these DevOps tools to help document, validate and automate a steady flow of software change along its path to end-users.

Here’s the problem DevOps tools are beginning to face: Like the EIS systems of the 60’s, fragmented DevOps tools have little or no visibility into the overall end-to-end process; however, they do generate lots of important data that is often locked away and isolated. This isolation creates a clear barrier to efficiency, visibility and agility across the software delivery process. Because of the limited function each individual DevOps tool performs, none have the gravitational pull required to define the larger data model. As comprehensive enterprise software delivery platforms emerge elsewhere, standard DevOps tools will face ever-increasing pressure to fold inside them.

Currently, Application Development Lifecycle Management solutions provide a platform to manage development projects, team resources and all manner of development activities. ADLM platforms also contain the “master production schedule” for every development initiative – past, present and future. The data contained within ADLM is now at the core of a quickly emerging software delivery data model and leading ADLM vendors are expanding their footprint pwell beyond traditional use cases. When it comes to ownership of this software delivery data model, I see no other solution category across the entire ecosystem with enough enterprise clout to pose a serious challenge to leading ADLM vendors.

Unified Software Delivery Platforms Are Already Emerging

Each of the five current ADLM leaders (according to Gartner’s most recent Magic Quadrant) are now racing to bring to market an enterprise software delivery platform that integrates many key DevOps capabilities.

Here’s my two cents on each…

The Giants in the space – IBM and Microsoft both have plenty of muscle and IP today. Clearly both are moving down the path toward a comprehensive software delivery platform. IBM acquired DevOps vender urban{code} several years ago and is hard at work building their developerWorks platform. Seemingly everyday, Microsoft is adding some kind of DevOps capability into its Visual Studio product suite. Still, I don’t see either vendor gaining much traction outside of their traditional (albeit very large) customer base. Perhaps more importantly, neither seem to have bonafide credentials within the super-influential agile development community and I believe this kind of street-cred (at least for now) is a must-have to dominate this space.

Atlassian does enjoy wide support among the agile community and no doubt has the broadest adoption footprint of any of the current ADLM leaders. Atlassian is in a strong position to mount a serious threat. However, Atlassian’s core product (JIRA), is widely believed to lack heavy-weight depth in the ADLM feature spectrum and it is often implemented as a departmental or “team tool”. They’ll have to develop deeper strategic planning and multi-team project capabilities to beat the rivals.

This May, software giant Computer Associates announced a definitive agreement to purchase ADLM heavyweight Rally and its agile development platform. In its announcement, CA said it intends to leverage Rally’s capabilities to “complement and expand CA’s strengths in the areas of DevOps and cloud management”. With the crucial addition of Rally, CA is now in a strong position to assemble its diverse capabilities into a single unified and enterprise-caliber software delivery platform. Now… can they seamlessly integrate all of the pieces-parts into a cohesive solution with a unified data model? If so, how long will it take?

Finally, I believe VersionOne may have a slight edge over the other ADLM vendors in the race toward a unified software delivery platform. I may be a bit biased because of my direct involvement in a joint project currently underway – none the less, here are four reasons why they will absolutely be a dominant force to recon with:

Vision: Robert Holler, VersionOne CEO, is clearly buying into the “enterprise software delivery platform” vision. He and his team have a well thought out strategy and they are actively executing against that strategy.
DevOps Automation: VersionOne has partnered with ClearCode Labs and both teams have been hard at work integrating ClearCode’s Continuous Delivery Automation framework into the VersionOne core product. This integration provides VersionOne the ability to orchestrate virtually any DevOps tool or platform and (just as importantly) incorporate all related data across VersionOne’s product suite to fee its quickly expanding data model.
JIRA Integration: VersionOne has just announced a tight integration into the JIRA platform. This integration will give them the ability to fold fragmented JIRA installations across the enterprise into the unified VersionOne platform providing a more strategic and enterprise-grade solution.
Availability: VersionOne’s automated delivery platform is available now and they are demonstrating their comprehensive solution to the eager agile community this week at the sold-out Agile2015 conference in Washington, DC.

Summary
The top 5 ADLM vendors are already well on their way toward developing enterprise-grade software delivery platforms that will consume many of the current “DevOps” automation solutions. Soon, development organizations will benefit from a comprehensive platform that can deliver increased efficiency, visibility and agility when compared to the heterogeneous solutions that have been cobbled together today.

About the Author/Dennis Ehle

Dennis Ehle, is a pioneer and thought leader in continuous delivery automation and agile delivery methodologies. Dennis is passionate about helping agile teams dramatically reduce the transaction cost associated with delivering incremental change. His company, ClearCode Labs, does just that by helping organizations continuously deliver high-quality software releases more frequently merging proven methodologies with empowering tools and technology. Twitter: @DennisEhle

August 4, 2015
Will DevOps Kill IT Outsourcing?: Part 1

Part 1: DevOps and outsourcing like oil and water

After nearly two decades of steady IT outsourcing for the sake of savings and efficiency gains, the shine seems to have worn off from this once glittering phenomenon. And according to many experts, DevOps and continuous delivery practices are playing a big part in many enterprises’ plans to reverse course on outsourcing key functions in their technology stack.

“DevOps transformations emphasize collaboration, which is often at odds with outsourcing arrangements,” says Jeff Gallimore, founding partner at Excella Consulting. “Outsourcing arrangements often reinforce stovepipes and silos rather than break them down. And silos are speed killers and barriers to improvement.”

According to a report out earlier this spring by outsourcing research firm Information Services Group (ISG), average outsourcing contract values saw their worst first quarter in a decade, dropping by 27 percent year-over-year. Big deals over $30 million were particularly hit hard, declining by 25 percent in both number and value.

Even without the tidal wave of Agile and DevOps practices seeping into enterprise IT over the last few years, many organizations are already predisposed to take a second look at their outsourcing contracts, says Gallimore, who sees time and again that many businesses aren’t getting the value they thought they would from their outsourcing deals.

“I hear many more negative experiences organizations have had with outsourcing arrangements than I do positive ones,” he says. “Poor quality. Higher-than-expected management overhead required. Inflexibility. Process challenges. All are reasons cited for why the arrangements didn’t live up to expectations.”

The anecdotal stories are also piling up of big name enterprises pulling more of their IT resources back in house after years of outsourcing. For example, General Motors made waves three years ago when its CIO Randy Mott said he would completely invert the outsourcing ratio the car company had by 2017. The firm continues to make headway in changing from 90 percent outsourced IT to 90 percent insourced, with many of the positions advertised these days focused on DevOps and innovation-related roles. Similarly, David Smoley, CIO of pharmaceuticals firm AstraZeneca, went public with his goal to shift the company from 70 percent dependence on outsourcing to 30 percent, along with a commitment to Agile and lean development.

As organizations increasingly witness the business boosts gained through a more collaborative DevOps delivery model, they are seeking ways to adopt continuous delivery practices across their entire software delivery lifecycle. But they’re finding that these practices are often wholly incompatible with the outsourcing arrangements they’ve developed, says says Jonathan Braunhut, chief scientist at KEMP Technologies. As he explains, many organizations will need to “unwind” these arrangements if they want DevOps to work well because in many cases the contracts and SLAs that govern them are inextricably tied with waterfall development.

“Outsourcing in an enterprise setting is often implemented in a way that advances outdated waterfall-like practices and processes that are antithetical to continuous delivery,” Braunhut says. “The very wall between dev and ops that the DevOps movement seeks to eliminate can become codified in outsourcing agreements, as a reasonable means of achieving an enforceable and accountable contract.”

More specifically, outsourcing tends to fly in the face of DevOps’ lean ethos, which is to “minimize handoffs and waste and increase quality by thinking about the overall system and flow,” says Gallimore, who believes that traditional contractual arrangements simply can’t operate with the kind of agility necessary to establish speedy and iterative cycles of continuous delivery.

Perhaps even more fundamentally flawed in the outsourcing relationship is the idea of ownership. As many organizations finally recognize the value that IT brings to the table—more likely now to serve as a revenue generator than a cost center—it doesn’t make as much sense to give up so much ownership over the technology and tech-enabled services that could potentially transform the business.

“To be most effective, enterprises need to own the transformation, and it has to be a truly collaborative effort across disciplines–business leadership, engineering, system administration, security & compliance,” says Thomas Enochs, vice president of customer success at Chef. “Collaboration and transformation are difficult to achieve with outside third parties or vendors. They need to own and drive the change themselves, and not be dependent on others.”

August 3, 2015
The Whole Nine Yards with DevOps and Agile Operations
Growing up as a boy in Manchester, UK, football (ok, soccer to US readers) was everything to me. I vividly remember being herded into the stadium stands on cold rainy Saturday afternoons to watch my team (#MCFC) win, draw or lose. Back then, football was for the die-hards – no frills or fancy tech, just hot tea and a suspect meat pie at half-time.

How things have changed. Now sporting events have undergone a complete transformation – all digitally-driven.

High Tech Sports

Take Levi Stadium, home of the San Francisco 49ers for example; considered by many to be the most high-tech stadium in the world. There’s too much tech to describe in detail, but you can be sure that all the Wi-Fi beacons, 400 miles of cable, photo voltaic energy and a funky mobile app have been designed with one thing in mind – enhancing the fan experience. So no ‘cattle-herding’ at a 49ers game – nope, a personalized digital experience that’ll find you your seat, food concessions, the closest bathroom — and of course the shortest beer line (#1 feature in my opinion).

Levi Stadium isn’t just another example of technical wizardry. It’s more an exercise in an organization deeply connecting with its customers through the delivery of highly experiential services. Understanding the needs, motivations and desires of their loyal fan base and servicing them accordingly.

It’s a pity this same thinking isn’t always applied in IT Operations – that is, deeply understanding what Ops ‘fans’ need if they’re to ensure a good business user and customer experience across new highly dynamic agile environments.

High Stakes IT Operations

Similar to the Manchester game day experience years ago, IT Operations has often done it tough. They’ve worked in isolated NOC’s and service centers, been placed on 24/7 call; working weekends and missing their kid’s birthdays. They have been served specialized ‘team merchandise’ in the shape of complex tools and processes – always counting on their loyalty to save the day when the next mega release comes over the wall. No great surprise then that Operations in many organizations is seen as uncommunicative and uncompromising – almost, well – tribal.

But IT tribalism doesn’t win matches in the software-driven economy game. Now, digital business realities means IT Operations must supplement its traditional fixation on reliability and resilience with a willingness to drive change – Operations must itself become more Agile. But how do you start? Thinking back to the innovation at Levi Stadium, here are three good ways.
1. Gaining Loyal Supporters – Development and Operations aren’t enemies, but like rival UK football team supporters the teams are often segregated. Dev and their tools to the left, Ops with theirs to the right. From a business perspective this is plain dumb – especially when it comes to getting the most value from technologies like application performance management. Rather than frantically monitor applications as problems manifest in production and then call Dev for help, wouldn’t it be better to incorporate operational intelligence much earlier in the software lifecycle? So when code is released, monitoring could immediately provide development with an early warning on operational issues. Conversely, Ops practitioners gain accurate and timely insights into production performance requirements and KPI’s – that’s not only a DevOps style win-win, it’s an agile like best practice to increase quality and avoid technical debt.
1. Filling the Business Stadium – The goal of Operations has always been keeping the business lights on, but Agile Operations now means making them shine brighter. Cloud, Microservices architecture and containers all give businesses the technical clout needed to support speed at scale, but it comes at a cost. Today’s sophisticated applications rely on specialized technologies, so it’s perhaps understandable that Operations chase the technology down the proverbial rabbit hole with more specialized tools. But the real truth is that any monitoring system which doesn’t deliver true business value by providing unified insight into how customers are actually experiencing IT services is of no use to anyone. Remember too that technology scale is about future proofing your business, so look for scalable monitoring to match it.
1. Delivering a Complete Experience – The digital experience for a 49ers fan doesn’t start and end on game day. It is delivered by a linked series of services – a digital journey comprising seat selection, ticketing, parking, entry, action replays, and player stats reviews – every one of which can positively or negatively impact the fans experience. Whatever your industry, IT Operations will be called upon to support similar services, meaning deep and uninterrupted visibility across the myriad of front and back technologies underpinning every customer interaction, engagement and transaction will become essential.
Successful digital transformation needs more DevOps and agile operations, agile development, faster release cycles and higher quality software. It’s time for IT Operations to stop fixing production problems and become an active participant in supporting these goals. Without Agile Operations, your role in the software-driven business game ferociously being played out will be like that meat pie I ate many years ago – suspect.
July 15, 2015
The Next Wave of Software Delivery – Integrating Agile and DevOps for the Digital Enterprise
In the past five years business cycles have greatly accelerated, forcing companies to respond faster to customer needs and competitive demands. This trend has been driven by business adoption of consumer-oriented technologies like social media, smart phones, b2c internet services. We all expect to have a constant stream of new services and capabilities delivered to us quickly and with little disruption.

There is currently intense competition across all industries, creating a digital “survival of the fittest” business environment. If a business function can be enhanced by a digital solution, it will be. Most enterprises are now dependent on the rapid implementation of new digital technology and the software solutions to remain competitive.

This extraordinary change has been achieved in part through adoption of Agile development, which promotes close collaboration between business users and developers. In Agile, environments, software development is broken into well-defined intervals with the goal of releasing software quickly, generally within less than a month. Updates are ongoing and nearly continuous, allowing product development teams to pare delivery time by 50 percent or more.

Adobe is a good example of this new way of doing things. After deploying CloudMunch DevOps platform Adobe has increased the pace of application development by as much as 60 percent.

Already, companies like Twitter, GameStop and Etsy use Agile processes as part of DevOps for continuous innovation. In fact, in a recent IPO prospectus, Etsy noted it updates code every 20 minutes, or 70 times per day. In 2014, the company executed 10,000 code “deploys”—a number that was inconceivable just a few years ago.

However, there is a disconnect between traditional IT processes and new Agilde and DevOps delivery processes. Here are three barriers to integrating Agile and DevOps in the enterprise:
- Traditional operational IT processes are much longer than intervals used in Agile. While IT is often planned and measured in quarters on an annual budget cycle, Agile development may happen on a two-week sprint. Once built, a business group is eager to use these new capabilities, but most IT organizations aren’t prepared to deploy new capabilities that quickly.
- The supporting software ecosystems are very complex, since applications are built by standing on the shoulders of giants. The comprehensive features supporting software allow development teams to focus on the desired business capabilities rather than low-level utility components. Since all of these components are updated frequently, the process of tracking and deploying the right supporting ecosystem presents a complex problem.
- Agile teams have autonomy in the use of development tooling, which results in a number of different and disconnected development tools being used. This also makes it difficult to implement these processes and provide the metrics that provide a consistent view of the enterprise portfolio of development projects.
Using modern tools for success in today’s hybrid cloud

To support rapid implementation of infrastructure needed to support the Agile delivery process, enterprises are increasingly moving more of their essential business operations to the cloud. For example, Adobe has embraced CloudMunch, a DevOps platform to automate delivery of software updates to dozens of its products, pushing multiple updates more quickly and easily to its customers.

Today, most companies have a hybrid IT infrastructure, comprised of both on-premise and cloud systems. While they have moved many functions to the cloud to take advantage of the benefits of cloud computing, they still keep many systems in-house. In the past, they’ve used development tools like Chef and Jenkins to manage application development and infrastructure automation, but these aren’t enough in today’s hybrid environment. Increasingly, businesses need to plug in and integrate their developers’ chosen tools into a single workspace to apply Agile development to a DevOps environment with a single dashboard. This will be critical if enterprises need to do Agile and DevOps at scale

This ability to manage Agile development in a hybrid cloud environment will become even more important in the next decade. Already, new platforms have emerged, allowing development teams to better orchestrate multiple toolsets & processes in a single “pane of glass” to streamline collaboration and application delivery. To stay competitive, businesses should assess their development infrastructure to ensure they don’t get left behind as technology progresses.

The impact of these accelerating changes in virtually every industry is astounding, affecting every facet of an organization, from company culture to management styles and business infrastructure. This is driven by changing customer expectations — breakneck speed and complexity have become the new normal. Companies are being forced to seek new development methods and leverage the cloud to create software that has the capacity to continuously adapt to ever-changing user needs.

These new approaches mean companies need to intensify collaboration and address the disconnect that exists between the old and new worlds. Successful integration of DevOps and Agile development will play a key role in the success of companies in today’s digital economy. In the new landscape, it’s about rapid, continuous high quality software delivery for the digital world, and companies that can do this at enterprise scale will outpace those that struggle to adapt.

About the Author/Pradeep Prabhu

Pradeep Prabhu, Co-founder and CEO CloudMunch, Previously Pradeep was Vice President and Head of SaaS for Infosys, where in his 18-year career he was instrumental in growing the tech company to a multi-billion dollar business. Before pioneering Infosys’ SaaS business, Pradeep incubated the Infosys-Microsoft solutions and alliance program and the Microsoft Technology Center. His previous leadership roles include establishing the Infosys Seattle office and delivering many successful software programs for marquee clients like Amazon, Boeing and Microsoft and conceptualizing and launching the internet banking product eFinacle, which today is installed in banks around the globe. Follow on twitter @cloudmunch
July 9, 2015
Appster Must Produce Mobile Apps Using Nimble DevOps Practices (It Has No Choice)!

In order to meet the many challenges of producing loads of different app/server projects large and small in extremely rapid succession, Appster collaborates on, automates, and dashboards everything it can. “Unlike traditional product companies (i.e. one product with lots of developers) or corporate IT shops (i.e. silos of products and services that rarely change), Appster has to handle a wide spectrum of software projects across a myriad of industries and vastly different client profiles,” says Martin Halford, CTO, Appster. Many of the projects Appster works on are mobile apps.

To do this, the DevOps tools and processes must tackle each new and unique development project in a continuous project stream with the same fervor that any large development project requires. “Our DevOps focuses on automating testing, builds, and deployments for highly custom, highly complex app/server software development projects,” says Halford.

The tools that help Appster to accomplish this include the XCode IDE, the Eclipse IDE, Atlassian’s BitBucket, SourceTree (for source control), and Jira (for Agile project and issue tracking), Jenkins for build automation, Amazon Web Services, and Klipfolio for dash-boarding, lists Halford. “And due to the short turn time and short duration of app/server projects—between 6 to 16 weeks—Appster does not at present practice Continuous Integration,” says Halford.

Appster has three main objectives: to deploy only from BitBucket, to protect the non-development servers from the developers, and to display the build statuses via dashboard to all. Deploying only from BitBucket holds the source code repository accountable as the “single source of truth”, says Halford. In order to protect the non-development servers, developers do not have root level access to the testing, staging, or production servers, only to the development server in this four server stack, Halford explains. Further, the QA Team has access to the development and test servers as well as limited access to the staging server, but no access to the production server. Only the Dev Ops Team / Linux Admins have access to the production server; they also have responsibility for the maintenance and administration of the entire server stack, Halford says.

App Dev At Appster Prior To DevOps

Prior to moving to DevOps approaches, Appster developed apps using manual steps and documented processes that were prone to human error, says Halford. “The process was also prone to human shortcuts and attempts to hot fix builds by circumventing source control and documented protocols such as those that demanded that no one make direct updates to the production server,” Halford says. These issues were not immediately uncovered.

A Typical Trip Through Today’s Appster Development Sprint

CTO Martin Halford shared his take on a round trip through Appster’s development process today. Appster will first generate development servers from saved generic Linux server images, which the Dev Ops Admins will also secure. Servers will ultimately include development, testing, staging, and production servers (or DEV, TST, STG, PROD). Appster will then generate source repositories in BitBucket, and Jira Projects that it will link to BitBucket. Appster will also create a product backlog using an Agile Scrum Software Methodology.

According to Halford, the development team will begin to develop software using Sprints, and each Sprint will follow this process:

A. The developers will check out their code and associate tasks with User Stories and / or Bugs in Jira.
B. They check code in as Changesets that are attached to tasks, which are in turn attached to User Stories or Bugs.
C. Developers automate and deploy builds as needed to the development server.
D. At the completion of a Sprint, the QA Manager selects completed tasks (User Stories/Bug Fixes) for deployment to the test server.
E. BuildMaster initiates the build, which pulls Changesets from the source repository, generates the build, and deploys it to the target server for testing.
F. After a successful Smoke Test, the QA team performs more involved tests to verify the build.
G. BuildMaster follows a similar process to deploy the build to the staging server for User Acceptance Testing.
H. The next Sprint begins and the cycle repeats itself.

Concluding Thoughts

“DevOps can heighten the effectiveness of mobile development by forcing and enforcing discipline on all parties, preventing human error, speeding QA testing cycles, effectively stopping defects, and highlighting inefficiencies in the existing development process such as could be alleviated through additional developer training and mentoring,” says Halford. Further, having a dashboard that is visible to all parties providing transparency into server issues so that Dev Ops Admins can respond immediately during build or production is critical to the DevOps process.

July 8, 2015
Best Practices to Prevent Scrum Failure
In my years of experience as a scrum master, I’ve encountered numerous situations where scrum has resulted in less-than-ideal outcomes, a Scrum failure if you will—not because the agile framework is flawed, but because the company didn’t understand, or disregarded, how scrum is supposed to work.

For example, I was once a scrum master for an e-commerce website production team where a new feature was considered so critical that it was prioritized over everything else in the backlog. Minimal time was taken for the development of user stories and my team had to make assumptions because the business stakeholders were unavailable. When we were nearing completion, the stakeholders came out of hiding and identified more business rules that required not only additional work, but some re-work of things that were already completed, including global changes to the company’s customer-facing website. The impact to the business had not been appropriately analyzed, and when the new feature was rolled out, the value delivered was negative, causing refunds and ultimately financial loss.

Sadly, scenarios like these are not uncommon. The scrum framework is ideal for situations where priorities change often, but many companies struggle with finding the right rhythm to implement and manage those changes. Scrum fails when companies don’t build a shared consensus on which principles should form their definition of agile delivery, or fail to educate everyone involved on how the agile processes should work. One very common example involves the most basic of agile artifacts: the sprint plan. I can’t tell you how many times I’ve completed a long day of planning only to run into an executive who wants to add something to the sprint. This is where understanding the methodology and sticking to it are important, especially because changing the sprint plan will impact your team’s morale, as well as its productivity. As much as we love our stakeholders, they should not swoop in at the last minute and bypass the product owner.

For those unfamiliar with scrum terms, the product backlog identifies the longer term priorities for a project or product team. The sprint backlog determines current sprint priorities and sets a shared goal for delivery team execution. Product backlog priorities can (and should) change all the time, but sprint backlog priorities should be locked down for the duration of the sprint. At the beginning of each sprint the team participates in a planning meeting where the highest priority stories are pulled from the product backlog and reviewed. The final output of a planning meeting is a prioritized sprint backlog that represents the plan for the sprint. Once the sprint plan is determined, it should be protected. There are always exceptions, but to ensure scrum works, the scrum master must protect the plan as much as possible.

So, how can you make sure your team moves as fast as possible but can still address changing priorities? Below are a few lessons I’ve learned:
1. Determine the Appropriate Sprint Length
First, and most importantly, determine a sprint length that is appropriate for your organization. I would recommend that companies new to scrum should avoid one week and four week sprints. One week sprints move too fast for new teams and are too heavily burdened with meetings. Four week sprint teams have a hard time staying with the sprint plan, especially new teams who don’t fully understand the impact of changing priorities mid-sprint. I’m not saying that four week sprints never work, but I find that a four week sprint length makes it difficult to keep the team on track. Two week cycles have traditionally been the preferred length for agile enthusiasts. When I’ve worked on project teams where code will not be moved to production at the end of the sprint, a two week sprint duration works well; however, my production teams seem to prefer three week sprints because they can develop new features in the first two weeks and use the last week to do production preparation work. Whatever length is chosen, it needs to work with your product environment. For example, if you’re currently running three week sprints but your scrum master is constantly having to battle to protect the sprint plan, then maybe a two week plan would be more appropriate.
1. Educate Your Stakeholders
Once the organization finds the appropriate sprint length, educating stakeholders is essential. How can we expect stakeholders to work with implementing scrum when we don’t help them understand how it should work? Actually communicating the sprint calendar is a good first step. Stakeholders also need to understand the product owner’s role and how they will assist them in assigning their requests the appropriate priority in the product backlog. This can be quite beneficial as many stakeholders who desire to add to the product backlog don’t realize that other higher priority features may not have been completed yet. Working with the product owner provides an impact analysis for the overall product. Sometimes it’s as easy as saying, “Is this more important than x?” Just that simple question has saved me many times as a scrum master. Sometimes stakeholders just want to get the request on the backlog and don’t mean for everyone to stop work on current priorities. If the request is determined to be the highest priority item at the time, then analysis of the desired behavior can begin. By the time the next sprint planning meeting happens the team will have a much better idea what the request is about and will be more likely to complete the feature with the value envisioned by the requestor.
1. Build Trust
Lastly, and perhaps most importantly, the team must build trust among the stakeholders. This has had a huge impact on my scrum teams and has stopped most emergency requests. Teams build trust by setting realistic sprint goals for themselves and then accomplishing those goals on a consistent basis. If the stakeholders know what is planned will get done, they have much more confidence that their request will be completed in a timely manner. This is not an easy task but can be accomplished by improving sprint planning.

The most effective way to improve sprint planning, and one that I find is often ignored, is utilizing the sprint retrospective. This is where the entire team reviews their performance during the sprint and discusses what went well and should continue and what areas need improvement. The team then decides what one or two things will make the most impact on the next sprint and brainstorm for ways to improve. The retrospective works well because the team decides what to change and begins to build a culture of continuous improvement. As far as agile benefits go, this is one of the most impactful to organizations.

Another way to build trust is to communicate changes to the plan early and honestly. Let stakeholders know why there was an adjustment to the plan. Was it due to a bad estimate? A delayed approval? Changing priorities mid-sprint? Once stakeholders start seeing the impact they have on the progress, they can assist the team in moving faster too.

The Result

When properly implemented, scrum can realize amazing benefits, but scrum teams and stakeholders have to work together to understand and implement the scrum procedures.

Here’s another true story: When I was a scrum master for another website project team, we had to halt development based on a third party delay. Because we were using scrum, this was no problem—we closed out the last sprint, complete with a proper demo, and the next day held our sprint planning meeting. We started a completely different project that was estimated to take approximately three months, matching the estimated delay for our primary project. We worked with a new product owner and successfully implemented the new feature as planned. The revenue generated from the feature was estimated to pay back the development costs in less than a year, and when completed, the team was able to return to the primary project.

That’s how scrum is supposed to work.

About the Author

Kathy Loveless is an AIM Consultant specializing in agile coaching, agile project management, scrum process improvement, and strategic development. She has an impeccable background in effective agile coaching and transformation and is passionate about immaculate communication and delivering results. AIM Consulting’s expertise in application development and product and project management covers the entire software lifecycle, from architecture to optimizing processes that ensure high quality outcomes with the most efficient delivery. To get in touch, tell us about your challenge.
July 6, 2015
SaaS: Building Tools for the DevOps-Minded

DevOps, Continuous Delivery, Continuous Integration and Continuous Testing are no longer just methodologies and development strategies — they’re all a part of a growing economy within the tech industry.

Great Software-as-a-Service (SaaS) companies such as Chef, Docker and New Relic have done an amazing job of spotting the trends early and building their tools and services around a new wave of development processes. Chef and Docker alone have reportedly raised nearly $250 million, and New Relic went public in 2014.

In fact, this very website is another example of how impactful these new development practices are — as several specialized news outlets have been born out of these revolutionary development strategies. So, not only has the DevOps methodology helped developers create better products, faster; but the process has opened opportunities for technical experts to share their experiences on sites like this, e-books, webinars, conferences and any other way people communicate.

On devopsy.kinsta.cloud, you’ll see the best articles on the web discussing how DevOps is defined in different organizations, why Fortune-500 companies are shifting old practices toward CD and the best practices of CI from industry experts.

In the race to be the most DevOps-friendly solution — no matter the niche — hundreds of startups have been founded and funded, which has only enhanced the toolboxes of thousands of developers. You’d be hard-pressed to find a good SaaS company that doesn’t offer integration with Jenkins CI, TeamCity or Chef, and that’s because not only should a SaaS product solve an important problem, but it should have minimum disruption of your current development process.

With all that in mind, Continuous Testing is one of the most exciting spaces in technology today. Continuous Testing is a process that aims to synchronize testing and QA with dev and ops, which ultimately yields better products more quickly.

There are several factors that make Continuous Testing successful in a team’s software development lifecycle — such as open-source collaboration solutions, test automation APIs and tools to track the historic performance of applications .

With the release of Load Impact Version 3.0, the global performance and load testing SaaS company has announced its latest feature that fits right in with the DevOps mindset: Performance trending.

Performance trending allows users to plot runs of the same test over time. That gives companies a clear view into data that locates patterns of performance degradation or improvement in order to easily validate code and infrastructure changes.

In the spirit of Continuous Testing, DevOps, CD and CI — performance trending improves users’ ability to gain both a low-level and high-level view into the performance of their systems. And with the increased number of tests run by DevOps professionals, performance trending yields valuable information for companies concerned with their application performance — which should be everyone.

Between devopsy.kinsta.cloud and plenty of great conferences and meetups around the world, it’s safe to say this economy of DevOps-minded tools will only continue to grow in the coming years.

July 1, 2015