devopsy.kinsta.cloud

Category: Enterprise DevOps

The Next Wave of Software Delivery – Integrating Agile and DevOps for the Digital Enterprise
In the past five years business cycles have greatly accelerated, forcing companies to respond faster to customer needs and competitive demands. This trend has been driven by business adoption of consumer-oriented technologies like social media, smart phones, b2c internet services. We all expect to have a constant stream of new services and capabilities delivered to us quickly and with little disruption.

There is currently intense competition across all industries, creating a digital “survival of the fittest” business environment. If a business function can be enhanced by a digital solution, it will be. Most enterprises are now dependent on the rapid implementation of new digital technology and the software solutions to remain competitive.

This extraordinary change has been achieved in part through adoption of Agile development, which promotes close collaboration between business users and developers. In Agile, environments, software development is broken into well-defined intervals with the goal of releasing software quickly, generally within less than a month. Updates are ongoing and nearly continuous, allowing product development teams to pare delivery time by 50 percent or more.

Adobe is a good example of this new way of doing things. After deploying CloudMunch DevOps platform Adobe has increased the pace of application development by as much as 60 percent.

Already, companies like Twitter, GameStop and Etsy use Agile processes as part of DevOps for continuous innovation. In fact, in a recent IPO prospectus, Etsy noted it updates code every 20 minutes, or 70 times per day. In 2014, the company executed 10,000 code “deploys”—a number that was inconceivable just a few years ago.

However, there is a disconnect between traditional IT processes and new Agilde and DevOps delivery processes. Here are three barriers to integrating Agile and DevOps in the enterprise:
- Traditional operational IT processes are much longer than intervals used in Agile. While IT is often planned and measured in quarters on an annual budget cycle, Agile development may happen on a two-week sprint. Once built, a business group is eager to use these new capabilities, but most IT organizations aren’t prepared to deploy new capabilities that quickly.
- The supporting software ecosystems are very complex, since applications are built by standing on the shoulders of giants. The comprehensive features supporting software allow development teams to focus on the desired business capabilities rather than low-level utility components. Since all of these components are updated frequently, the process of tracking and deploying the right supporting ecosystem presents a complex problem.
- Agile teams have autonomy in the use of development tooling, which results in a number of different and disconnected development tools being used. This also makes it difficult to implement these processes and provide the metrics that provide a consistent view of the enterprise portfolio of development projects.
Using modern tools for success in today’s hybrid cloud

To support rapid implementation of infrastructure needed to support the Agile delivery process, enterprises are increasingly moving more of their essential business operations to the cloud. For example, Adobe has embraced CloudMunch, a DevOps platform to automate delivery of software updates to dozens of its products, pushing multiple updates more quickly and easily to its customers.

Today, most companies have a hybrid IT infrastructure, comprised of both on-premise and cloud systems. While they have moved many functions to the cloud to take advantage of the benefits of cloud computing, they still keep many systems in-house. In the past, they’ve used development tools like Chef and Jenkins to manage application development and infrastructure automation, but these aren’t enough in today’s hybrid environment. Increasingly, businesses need to plug in and integrate their developers’ chosen tools into a single workspace to apply Agile development to a DevOps environment with a single dashboard. This will be critical if enterprises need to do Agile and DevOps at scale

This ability to manage Agile development in a hybrid cloud environment will become even more important in the next decade. Already, new platforms have emerged, allowing development teams to better orchestrate multiple toolsets & processes in a single “pane of glass” to streamline collaboration and application delivery. To stay competitive, businesses should assess their development infrastructure to ensure they don’t get left behind as technology progresses.

The impact of these accelerating changes in virtually every industry is astounding, affecting every facet of an organization, from company culture to management styles and business infrastructure. This is driven by changing customer expectations — breakneck speed and complexity have become the new normal. Companies are being forced to seek new development methods and leverage the cloud to create software that has the capacity to continuously adapt to ever-changing user needs.

These new approaches mean companies need to intensify collaboration and address the disconnect that exists between the old and new worlds. Successful integration of DevOps and Agile development will play a key role in the success of companies in today’s digital economy. In the new landscape, it’s about rapid, continuous high quality software delivery for the digital world, and companies that can do this at enterprise scale will outpace those that struggle to adapt.

About the Author/Pradeep Prabhu

Pradeep Prabhu, Co-founder and CEO CloudMunch, Previously Pradeep was Vice President and Head of SaaS for Infosys, where in his 18-year career he was instrumental in growing the tech company to a multi-billion dollar business. Before pioneering Infosys’ SaaS business, Pradeep incubated the Infosys-Microsoft solutions and alliance program and the Microsoft Technology Center. His previous leadership roles include establishing the Infosys Seattle office and delivering many successful software programs for marquee clients like Amazon, Boeing and Microsoft and conceptualizing and launching the internet banking product eFinacle, which today is installed in banks around the globe. Follow on twitter @cloudmunch
July 9, 2015
“Show Me The Money!” – Delivering DevOps Value
HM Heath Solutions delivers new business applications in the healthcare industry to 35 million customers. In order to embark on their DevOps journey to improve software delivery, Valerie Scott, their Manager of Shared Services Division, had to show senior management an ROI on a successful project, before they would invest fully. On June 2, Val Scott along with Rick Slade of IBM joined Alan Shimel of devopsy.kinsta.cloud in a webcast to discuss this interesting client case study. During their discussion, they illustrated in vivid detail how HM Health Solutions showed success in DevOps to their senior executives, for them to buy in and sponsor the project.

Val and Rick’s webinar parallels many similar discussions I’ve had with senior executives. To net it out, the bottom line is, as it was so clearly and unequivocally articulated in this amusing clip from the movie Jerry Maguire: “Show Me The Money!”

When I meet with senior LOB, Finance or IT executives — they shout a similar refrain:

“All I want to do is buy a business outcome. I want to invest in an IT project, and have it pay back. Unfortunately for many application development (or other types of enterprise IT) projects, that pay back is illusory, and ROI difficult to measure.”

— Jane or John Executive, representing nearly every IT executive or CFO I have ever met

Unfortunately, many struggle to quantify, monetize and demonstrate the financial return on investing in IT projects. Yet as described in this blog, it is the primary responsibility and outcome of management; leading the economic performance of their department or unit. Show me the money! Fortunately, many clients experience positive returns, both operationally and economically. Ten (with links to more) client case studies with returns are highlighted in this blog, and summarized on this page.

As you undoubtedly have experienced yourself, senior executives respond to clear business value and benefits articulated as financial returns, as ROI. But how do you do that for DevOps, a set of technical practices and associated culture change? As Val and Rick described during their informative webcast, comparing productivity, time-to-market, speed of delivery, and quality, both before and after a DevOps pilot, are crucial. These operational metrics will be improved by adopting DevOps. However, to win the hearts and minds of senior executives, you have to monetize the results, in other words – show them the money! Technical techniques and practices such as those Val and Rick described to Alan, shifting left, test driven development, and automating application build, deployment, and test are absolutely necessary, but they are insufficient to win the argument with senior executives. The gains they produce are necessary, and impressive. The operational results have been stellar thus far at HM Health Solutions, per Val:
- a 67% reduction in resource demands on the release team & process,
- a 75% reduction in relative hours to repair defects per environment,
- an 82% reduction in application deployment time.
Yet as impressive as these operational results are, they are insufficient. As Val described, arguments about business value, technical value, or improvements to developer or operations staff productivity, alone did not persuade the management. The magic word is an economic and financial one: “investment”. Making a strategic investment to improve delivery speed and quality of business applications was the key to winning the hearts and minds of the senior executives.

In addition to the technical practices described above, Alan asked Val to describe the tools HM Health Solutions adopted. A continuous delivery solution with application release automation software from IBM UrbanCode was implemented, including application build, deploy, and release.

As a way of summarizing the webcast for the audience, Alan asked Rick and Val for their two key takeaways; the two most important messages and things to remember. Rick’s:
1. Knowing which metrics are critical to your organization. There are many benefits to adopting DevOps, and many potential hard numbers that are straightforward to capture and measure. Deciding which metrics have the most impact and are most persuasive to your stakeholders is critical to making the case to invest in DevOps.
2. Implementing tools and automation: automated testing, and linking that to build and deployment processes and automation.
Val’s: Rick’s two takeaways of course! And one more…
1. DevOps is new in the industry, and is not yet well understood or common practice in enterprise IT or among its leaders. Understand your audience, and become good at explaining it in straightforward and clear business and financial terms to senior executives. Especially persuasive are other client proof points, and the results of pilots in your own enterprise IT environment. Make sure pilot results show that the solution works and delivers timely business value, and that the related facts and metrics clearly quantify the results.
Read this blog to know more and Check out other customer proof points/ case studies

This was a very informative, fast paced, and information rich webcast. I hope that you find the replay useful in continuing your DevOps adoption journey.

If you’d like to explore more about management responsibilities and know-how for establishing an ROI for DevOps and app release & deployment automation, see this ROI web page, and read this ROI blog for additional resources on method, approach, making the case, and calculating the ROI. And please don’t hesitate to reach out to me on twitter: @paspung.

I look forward to hearing your success stories in making the ROI case to senior executives, and delivering DevOps value to your organization!

Peter Spung, DevOps Deploy and UrbanCode Integration Executuve, IBM Cloud, @paspung
July 2, 2015
Eight steps to a sustainable DevOps transformation
DevOps is not a standard. It is more a philosophy described by a loose set of concepts and tools, many rooted in the agile methodology and continuous delivery. That often causes confusion about the potential scope of DevOps, and makes it difficult to determine a definitive start and end point for a DevOps initiative. Moreover, DevOps requires significant cultural and organizational change, which can be hard to achieve and sustain.

So now we know what DevOps is, here we propose eight steps to a sustainable DevOps transformation, based on the use of a modified version of the Kotter change framework (1):
1. Develop a sense of urgency
“Deliver it faster and less expensively”. The two most common requests from the business—whether it is to deliver digital transformation, find more agile and cost-effective alternatives to internal IT, or other requirements.

IT organizations must become fully aware of the urgency to change in order to stay relevant for the business.

In the context of dev/ops this urgency is measured by release velocity, turnaround time for new features and other metrics. Benchmarking helps here: comparing delivery time results with other equivalent companies using DevOps tools, for example. Any deficiencies can be brought up through a value stream analysis with a TIMWOOD waste analysis.
1. Form a guiding coalition
For successful DevOps transformation, identify a team which is both supportive of the change and able to communicate progress. Include senior IT management from operations and development, plus key stakeholders in DevOps processes.
1. Create a vision
The DevOps vision for continuous delivery should be aligned with the business vision. It should be broad enough to cover all aspects of DevOps, but sufficiently narrow to guide decisions in the subsequent change process.

Gartner use the term “boundary objects” to denote things that can be “interpreted differently across different communities, but that posses enough common understanding to make them useful mechanisms to communicate with and improve knowledge”.[1] Boundary objects are thus considered a possible way to align on major concepts; the objects are flexible to local requirements, yet robust enough to maintain a common identity across sites. A DevOps/continuous deployment roadmap is also a necessity, encapsulating important themes and their priorities on an approximate timeline. Avoid detailed project plans, on the basis that DevOps should follow an agile approach and is all about continuous improvement.

Should the whole organization adopt DevOps or should DevOps remain limited to those areas in which the need for agility ultimately beats strong process compliance needs, risk mitigation and availability requirements?

There is no definite answer to this, but keep in mind:
- DevOps is a lot about culture. In the long-term it will be difficult to maintain a single organization, built upon two different cultures.
- IT systems are often inter-dependent; simple changes have to be implemented across an array of systems.
1. Communicate the vision
Communicate the vision, strategy and roadmap again and again. Moreover, start to live the cultural values of continuous delivery, such as avoiding blame, putting individuals before the process and promoting transparency.
1. Empower others to act on the vision
Many aspects of the vision—such as reduced process complexity and a revised understanding of collaboration—should already diffuse into the company culture even if “full” DevOps is not in place yet.

The guiding coalition needs to remove obstacles (such as too many process steps or approvals and rigid system access rights). These potentially limit the ability to experiment with new tools or methods. Indeed, any systems or structures that seriously undermine the vision should be replaced. Likewise, risk taking and nontraditional ideas, activities, and actions should be encouraged.
1. Create short-term wins
For successful DevOps deployment and transformation it’s important to deliver visible performance improvements, preferably a couple of pilot projects using established DevOps teams.

When choosing the pilot team, ensure:
- The team can be trusted to manage continuous delivery
- The application is a good fit in terms of architecture and technology
- The DevOps benefits are apparent to the business
1. Consolidate improvements and scale DevOps
Use the credibility derived from successful pilots to change systems, structures and policies that don’t fit the continuous delivery vision.

Have the different teams implement their own “flavors” of DevOps, best suited to their specific environment.

These teams can leverage the experience, processes and tools developed through the pilot, but are not required to do so as long as their implementation fits into the overall continuous delivery vision.

Avoid governance frameworks, which run contrary to the concept of agile.
1. Institutionalize new approaches
Articulate the connections between the new behaviors and corporate success with continuous delivery. To do this, you need the right metrics in place, linked to top-level KPIs. At this stage the entire business system has to be adopted to the new culture so that the change becomes part of “business as usual”.

This “institutionalization” also needs to be supported by metrics, incentive schemes and the appropriate style of management.

(1) John P. Kotter, The Heart of Change, 2002

[1] Gartner, How to Scale DevOps Beyond the Pilot Stage, Cameron Haight, 11 March 2015, https://www.gartner.com/doc/3003922/scale-devops-pilot-stage
June 29, 2015
Do/Should Developers Own Infrastructure?

In the pre-DevOps era, there wasn’t much question of developers owning infrastructure, if only because for the most part, they didn’t want to own it. They wanted infrastructure to work for them (or at least not get in the way), but they didn’t want to be stuck with the time-consuming and generally thankless task of maintaining and managing it. Add to that senior management’s understandable desire to keep infrastructure in the hands of trained IT personnel, and the IT staff’s equally understandable desire to maintain an orderly network, and the result was a relationship between developers and infrastructure that was cool and somewhat distant.

But with DevOps, we are entering (or have already entered) the era of infrastructure as code, and with it, the developer-infrastructure relationship has changed. If infrastructure is code, then almost by definition, infrastructure becomes to some degree a function of development, or at least so hard to separate from development that the distinction becomes almost irrelevant.

(more…)

June 25, 2015
Rework is Choking Software

Rework is Hell

“Software may be eating the world, but rework is choking software”, tweeted John Jeremiah (@j_jeremiah). To shed more light on what is choking software, new data was released last week in the 2015 State of the Software Supply Chain Report.

In its discussion of application quality and integrity, the report revealed that the average application includes 106 open source components. It is clear that the use of these components has benefited development tremendously in helping to speed time to market and improve innovation. While the benefits are undeniable, development teams are also delivering applications that are “insecure by design”. Of the 106 components per application, the report’s analysis revealed an average of 24 (i.e., 23%) have known critical or severe security vulnerabilities. Those same apps also showed an average of 9 restrictive license types (e.g., GPL, AGPL, LGPL).

By electively sourcing components with known vulnerabilities and potential license risks, software development teams are building up higher levels of technical and security debt. And in order to reduce some of that debt, unplanned and unscheduled rework is often required.

No Developer Intentionally Uses Bad Parts

I have never met a developer that intentionally wanted to use “bad” components in their applications. That said, I have also never met a developer that wanted to spend four hours researching the latest versions, open source licenses and known security vulnerabilities for components (including dependencies) they wanted to use in their applications. While 57% of development organizations have internal policies in place that direct “thou shall not use components with poor quality and integrity”, many have stated that the policies are not followed or are simply not enforced.

The problem is one of volume and velocity, while people clutch to old processes that simply can’t keep pace. The analysis in the 2015 State of the Software Supply Chain report revealed that billions of open source downloads occurred in 2014, of which 6.2% had known vulnerabilities (1 in 16 downloads). For some large companies, the volume of downloads exceeded 240,000 — where 15,000 (7.5%) had known security vulnerabilities. At these volumes, manual processes and paper-based policies to prevent poor quality or risky components from getting into your software have no possibility of keeping pace. While software development practices aim for higher velocity, through component-based development as well as Agile, continuous and DevOps practices, the processes to keep up with it have not changed enough.

The only way to keep pace with this velocity is through automation. And if we want to avoid unplanned rework associated with remediating known security vulnerabilities or selecting alternative components with approved license types, automation has to become the norm rather than the exception within development.

At Speed

“If I look at the mass, I will never act. If I look at the one, I will.” — Mother Teresa

Looking at the massive volume and velocity of open source and other artifact consumption can be discouraging and overwhelming. However, the volume and velocity within our software supply chains will not diminish—and without a new approach, the volume of unchecked quality and integrity of parts being consumed will continue to build up as technical debt.

Automation in areas of testing, build, and deployment has provided significant performance benefits. Likewise, investments in software supply chain automation have shown markedly improved efficiency and controlled risk, as the best practices in the 2015 Report illustrate. Automation can unleash the potential of an organization’s development capacity. Rarely is there such an opportunity to simultaneously increase speed, efficiency, and quality.

In order to improve the quality and integrity of components used in our applications, we need to release our clutch on old practices. We can further embrace automation across our software supply chains — and improve the quality of our applications — by considering these three steps:

1. Start with creating a software bill of materials for one application. Visibility into one application can help you better understand your current component usage. A number of free and paid services are available to help you create a software bill of materials within a few minutes. The bill of materials will help you to identify the unique component parts used within your application and the suppliers who contributed them. These reports list all components used, and several services also identify component age, popularity, version numbers, licenses, and known vulnerabilities.

2. Design approval processes to be frictionless, scalable, and automated. Manual reviews of components and suppliers cannot keep pace with the current volume and velocity of consumption. Your organization must not only define your policies for supplier and part selection but also find practical ways to enforce them without slowing down development or inadvertently encouraging workarounds. Policies must be agile enough to keep pace with modern development. Strive to automate policy enforcement and minimize drag on developers.

3. Enable developer decision support. Developers are primary consumers of components in software supply chains. They initiate every component request. Help developers by automating the availability of information on component versions, age, popularity, licenses, and known vulnerabilities within their existing development tools so it is easy to pick the best components from the start. By selecting the highest-quality components from the highest-quality suppliers early, you will improve developer productivity and reduce costs.

Here is the complete 2015 State of the Software Supply Chain Report. I will also be discussing more of the Report findings and best practices on a webinar scheduled for Wednesday, June 24, 12pm ET (register or view it on-demand).

The previous blog in this series was, Fewer and Better Suppliers. Next up, I will be reviewing Visibility and Traceability practices across the software supply chain with more commentary from the 2015 State of the Software Supply Chain Report.

June 24, 2015
Bimodal IT and Remodeling Traditional IT for Greater Agility
Bimodal IT is a Gartner concept created to help CIO’s understand that IT organizations need to support both traditional and agile modes of IT solution delivery and operation in order to make wise choices about infrastructure, process, people, and tools. Much attention has been given and rightfully so to the disruptive and innovative “Mode 2” teams building on the practices established by web-scale giants like Amazon, and Google. However, in enterprises there still exists a sprawling majority of “Mode 1”, traditional, industrialized applications run by teams that must focus on risk aversion, security and compliance. While these will likely not match the innovative velocity of agile IT teams and applications, traditional IT teams are still under pressure to get faster, more agile, innovative, productive and cost-efficient. Yet, the Mode 2 playbook can’t be applied wholesale to Mode 1 transformation work because it often assumes a greenfield situation—kind of like a new home construction. By contrast, Mode 1 modernization is more like renovating an existing home that must proceed in steps that address the present realities of applications, legacy infrastructure, business mandates, organizational interactions, and technical skills.

Understanding Bimodal IT

The Bimodal IT concepts is meant to distinguish between applications and teams that are traditional in their focus, and those that are agile. In her blog post on Bimodal IT, Gartner analyst Lydia Leong explains Bimodal IT thusly:

“Traditional IT is focused on “doing IT right”, with a strong emphasis on efficiency and safety, approval-based governance and price-for-performance. Agile IT is focused on “doing IT fast”, supporting prototyping and iterative development, rapid delivery, continuous and process-based governance, and value to the business (being business-centric and close to the customer).”

The reason that Gartner promotes the idea of Bimodal IT to CIO’s is to help them see that it can be dangerous trying to optimize processes, human resources, infrastructure, and tools to cover such different modes, and as a result, optimizing to neither. Rather, the right move according Gartner is to allow agile teams room to develop a brand new set of capabilities that can then help transform the whole organization. Lydia continues in her blog:

“We’ve found that organizations are most successful when they have two modes of IT — with different people, processes, and tools supporting each. You can make traditional IT more agile — but you cannot simply add a little agility to it to get full-on agile IT. Rather, that requires fundamental transformation. At some point in time, the agile IT mode becomes strategic and begins to modernize and transform the rest of IT, but it’s actually good to allow the agile-mode team to discover transformative new approaches without being burdened by the existing legacy.”

The Contrast Between Agile and Traditional IT

Traditional IT and Agile IT are different in a number of ways. Perhaps an effective way to explain the difference is to picture what these applications and teams look like.

The classic agile IT initiative in the enterprise is around a mobile, e-commerce or web-centric application that is aimed at moving into new markets, improving engagement with new customer segments, or disrupting a current market. The application is fully virtualized, and the preferred infrastructure is a public cloud such as AWS. The team is conceived as a unified, DevOps team of line of business product managers, developers, testers, and operations folks working collaboratively on an agile development methodology. Automation is built deeply into the process, with infrastructure as code tools used to manage public cloud virtual machines and related infrastructure components via well-documented RESTful API’s, plus other DevOps toolchain elements automating and managing most processes, including continuous integration, continuous delivery and continuous deployment of code into production.

Traditional IT stands in pretty stark contrast. Let’s consider an industrialized, mission-critical application that has evolved slowly for years. The application (which is usually composed of anywhere from 5 to 10 or more components) is likely running on a mix of legacy (think RISC UNIX), dedicated (x86-based) and virtualized servers (VMWare and perhaps Hyper-V) running in a private or hosted private datacenter, using a mix of traditional and software-defined storage and connected by Ethernet and Fibrechannel switches. The application components themselves may be legacy code, making them difficult to quickly port to more modern infrastructure. For example, I was talking recently with a colleague who told me about his sister who still writes code in Fortran for a major bank. With such a large, complex, long-lived and mission-critical application suite, the IT organization maintaining it and the developers creating updates and upgrades are highly concerned with quality, reliability and predictable performance. The nature of the overall team working on an industrialized application is a far cry from the sort of “two pizza” DevOps teams on the agile IT side. Rather, it’s more of a loose consortium of teams such as developers, testers, security and compliance, plus outsourced contractors and vendors. The development process is waterfall-based, managed by an application lifecycle management (ALM) suite of some sort. Automation is scant, and in particular it takes a long time to pull together infrastructure for those dev, test and other teams to perform their jobs. Despite that, a production-like environment for testing, security, compliance, etc. is considered necessary to minimize downtime and other quality risks. As a result, release cycles can stretch into many months.

Modernizing Traditional IT

When my wife and I remodeled our 1908 Craftsman-style house about 15 years ago, we discovered that an earlier renovation had actually cut out a third of a key supporting beam, leaving one part of the house literally hanging on a beam that was only attached to one side of the house. Needless to say, we were a bit alarmed to find that out. When dealing with mission-critical applications and infrastructure, you can’t be careless. As mentioned earlier, simply taking the playbook for a greenfield, agile IT application and trying to apply wholesale it to a legacy, industrialized application usually won’t work. Making traditional IT more agile requires a step-wise renovation—a risk-mitigated path that leads towards agile and DevOps practices, but without a great leap into the dark. The steps need to establish the building blocks to agile/DevOps practice without disrupting the application, without introducing downtime, and while delivering solid returns along the way. Successful legacy application and service ‘remodeling’ efforts observed in highly traditional organizations such as telecommunications and cable operators provides good ideas on how to take those steps:
- Identify or Procure Automation Skills: The initial steps to remodeling Traditional IT are about incrementally improving current practice with automation and at the same time making it ready for agile and DevOps. This means that to be successful, you need to get some automation skills into the mix. A team that can’t take hold of its own destiny in modernizing a Traditional IT application is not going to have a sustainable change, and that means identifying or procuring some automation expertise and protecting that person’s time to focus on the renovation rather than on firefighting.
- Private Cloud IaaS: One of the most serious obstacles to agility is the painful amount of time and effort it takes to allocate and provision infrastructure for developers, testers, etc. and the lack of standardized, consistent infrastructure environments, due to manual processes. This is exactly the issue that drives some development teams to go shadow IT and start developing on AWS. Often times, these development teams then discover that integrating that software into the production environment is plagued with translation issues. Nonetheless, the first order of business then is for the infrastructure team to establish IaaS that can serve up full production-like infrastructure environments to devs, testers and other constituents. This isn’t trivial when you’re dealing with a combination of legacy servers, bare metal x86, VMs, and perhaps networking, security and other devices.
The IaaS delivers a number of key improvements. First of all, it speeds up access because developers, security, compliance and other personnel can now get access to infrastructure in the space of minutes rather than the space of days or weeks. More than just speed, IaaS enables asynchronous productivity by various teams to happen 24×7, so that quality progresses in an even fashion, unimpeded by office hours. This is especially helpful if there are global teams, contractors or vendors involved. IaaS also provides standardization and consistency across all efforts. Finally, IaaS saves costs. Significant amounts of operations personnel time that was devoted to painstaking manual infrastructure assembly can be repurposed to higher value activities, and infrastructure utilization rates can easily double or triple almost immediately, which reduces the need to purchase more capital equipment.

A word here about GUI’s. One of the key notes of dissonance that often occurs is when people hear developers say that they don’t want GUI’s, they want API’s. That in my view is absolutely valid, but more so for Mode 2 Agile IT teams. It turns out that IaaS GUI portals can be rather helpful in providing an easily understood reference point to multiple teams that may have highly varying understandings of the end-to-end picture of the application and its infrastructure. For example, if an IaaS self-service portal provides a full visualization of the end-to-end infrastructure involved, a developer who might assume that latency is always near zero might become more aware that there is a WAN in the middle of the end-to-end infrastructure.
- Test as a Service: Many development and testing teams have taken some stab at test automation. However, test automation tends to live and be maintained at the individual level. The natural follow-on to IaaS is to build a standardized regression or certification of automated tests that can be offered from the IaaS portal—test as a service (TaaS). This allows teams to access production like environments, do their development work, then easily launch a comprehensive certification test. Tests naturally should report their results into the ALM tool so that individuals and teams can review results in near real-time to keep the process moving. The benefits are similar to IaaS: increased velocity, standardization, productivity and efficiency.
- DevOps Evolution: With a combination of IaaS and TaaS, the stage is now set to deploy a DevOps toolchain and start doing cross-functional technical collaboration, and agile methodologies. For example, the IaaS and TaaS can be launched by a unified continuous integration server upon software development check-ins from any team. Of course, DevOps doesn’t just mean tools and automation, but the process of coming around collaborative devtest infrastructure and common certification tests to build a common, agile development practice is one of the steps towards a more broadly responsive cycle.
Conclusion:

Traditional IT may not be the sexiest beast in the technology forest, but it’s often mission-critical. Gartner’s Bimodal IT concept encourages CIO’s to let go and let Mode 2 Agile IT teams do their thing to create new culture and practices. Translating that to Traditional IT requires a more step-wise approach. Progressing from manual processes to IaaS to systematized, self-service test automation as the foundation for DevOps evolution is a sound way to proceed. With these steps. Traditional IT can “do IT right” and do IT faster.

About the Author/ Alex Henthorn-Iwane

Alex Henthorn-Iwane joined QualiSystems in February 2013 and is responsible for worldwide marketing and public relations. Prior to joining QualiSystems, Alex was vice president of marketing at Packet Design, Inc., a provider of network management software, and has 20+ years of experience in senior management, marketing, and technical roles at networking and security startups.

Through his roles at QualiSystems, Packet Design, CoSine Communications, Corona Networks and Lucent Technologies he has acquired expertise in cloud computing and the opportunities presented through virtualization. He has written for Embedded Computing, Virtual Strategy Magazine, Datamation, SDN Central, Datacenter Knowledge and InformationWeek.

Connect with him on LinkedIn, Twitter, or SlideShare.
June 24, 2015
Bringing the Database into the DevOps Fold

In my last post on devopsy.kinsta.cloud I asked the question, “Where is the DBA in the DevOps Conversation?” I have wonderful news. I’ve found them! Having not been offered a seat at the table with everyone else, the DBAs have decided to have their own party in the form of a survey and corresponding report by Unisphere Research for the Independent Oracle Users’ Group.

Until now so much of the research, literature, and web content on the topic of DevOps has been completely devoid of talk about the database and DBAs. The oversight was significant and troubling given the immense strategic business value of any organization’s ever growing store of data. The IOUG report does an excellent job of framing the case for DevOps in terms of the special considerations required when trying to adapt to the lightning fast pace of change in modern IT.

For me, the biggest take away from the report is that DBAs and their managers are feeling the pain of increased release velocity acutely.
Interestingly, while the age of automation and cloud may be upon us, an increasing number of database managers and professionals report that they are being mired in manual processes. More than one-third cited this as a challenge, up a dramatic 162% from four years ago.
There’s no question that companies large and small understand that releasing quality services and features based on market demand at a faster rate than your competition keeps your organization on top. It’s also apparent that product development and most of IT operations have made the necessary investments in process modification and the automation necessary to truly achieve faster delivery. Unfortunately the report confirms what the good folks at my company, Datical, have been saying all along. The lack of safe, intelligent deployment automation options for the database is starting to limit the potential gains of DevOps implementation…and the data team knows it. But it’s not the actual script execution that takes so much time. It’s all the time spent before that maintenance window making sure that security isn’t being compromised, that the change is compliant with external regulations and corporate standards, and that it won’t negatively affect your data.

Today’s manual database operations, processes, and approval time are among the causes of delays holding up the real-time enterprise, many database managers and professionals admit. Frantic development cycles and the accelerated pace of business innovation requires that data and insights be available at a moment’s notice. Close to 41% of respondents report that it takes a week or more to approve change requests, as well as to configure new databases for key purposes, even for testing.

To safely reduce these cycle times a DevOps solution for database deployments has to harness the expertise and experience of an organization’s DBAs to fuel intelligent, environment-aware automated validation of these change requests. The automation can’t simply parse the SQL script looking for patterns that may be problematic. It must be able to evaluate all 5 W’s associated with the change and make the correct determination as to whether it is acceptable or needs further review. By focusing the valuable finite resource that is your organization’s DBAs on only those changes that need their special attention, you’ll reap even greater rewards from your DevOps investments.

I’ve just scratched the surface of what’s available in this report and I highly recommend you read and share it with the DBAs at your company. The survey covers responses from folks in every size company and at every level of the database team. It is full of facts and observations that will be readily familiar to any DevOps enthusiast or practitioner. DBAs, IT, Developers, and the business are now all speaking the same language when it comes to DevOps. Do what you can in your organization to make sure they are speaking that language to each other.

About the Author/Pete Pickerill

Pete Pickerill is Vice President of Products and Co-founder of Datical. Pete is a software industry veteran who has built his career in Austin’s technology sector. Prior to co-founding Datical, he was employee number one at Phurnace Software and helped lead the company to a high profile acquisition by BMC Software, Inc. His ability to understand product demands from a customer’s perspective and translate those demands into actionable product and development plans has led to expanded duties at every company for which he’s worked.

June 23, 2015
$The Simple Math of DevOps$

The Simple Math of DevOps
We are living in an information- infused society, much dependent on various technological innovations taking place all around us. In such conditions, as consumers become more connected, they are empowered with more information and choices. They expect quick and consistent information available at all times. Likewise, our business leaders expect their IT teams to build relevant applications and stay ahead of their customers’ needs with an excellent customer experience.

My IBM colleagues and I have been working very closely with major enterprise clients as they begin their DevOps transformational efforts. We’ve found few key points and messages that resonate very well with decision makers who need to set a new course for their teams in the wake of impact from mobile, big data, analytics, cloud, and the Internet of Things technologies.

So what is it about DevOps that is so appealing? First and foremost is the notion that DevOps is all about attaining Speed to Value. Speed is king right now because so many of the business leaders we work with are under the gun to deliver value as quickly as possible. Yet, because of the high cost of technical debt they have come to a state where speed has to be sustainable. It’s no longer easiest to go outside their organizations for the high priority key projects as mobile applications were done recently. They need a holistic approach that builds on their existing capabilities.

Well what does that have to do with math? Here’s the thing, everyone is trying to minimize the time it takes to deliver value and attain feedback.   We can think of the Time to delivery as the following equation.

T_DELIVERY = T_PLAN + T_DESIGN+ T_DEVELOP + T_BUILD + T_DEPLOY + T_TEST + T_FIX+ T_RELEASE + T_EVALUATE

In an optimal world or software factory that is fully automated the T_DELIVERY is optimized by minimizing the time for each of the tasks required to complete the delivery. Most organizations follow a SDLC that is based on this simple math. They estimate the time to do each part and then the total project plan is the sum of the parts. Further, they tend to define work in relatively large chunks or big-up-front style of work definition which inherently increases the time for each task (T_X). As a result, the industry has about a 50% (or worse) failure rate when it comes to completing a project within the estimated time. That causes a lot of hardship because the business can’t count on IT to deliver value to customers in the projected time frames let alone with speed.

Because application delivery is a set of very human oriented tasks, a key factor that will determine the speed to value is TRUST. It’s pretty obvious when you draw out the value stream mapping of how work gets done at a given customer site. As members of software delivery teams lose trust in the validity of the work as it flows through the lifecycle then a large amount of rework and waste is introduced. In mathematical terms our equation becomes:

That is, the Tasks we do in a delivery cycle are impacted by the degree of trust we have in the hand-offs from one to another. If we have zero trust then our T_DELIVERY will be infinite (divide by zero). 100% trust and our T_DELIVERY will be only limited by how fast each task can be performed.

Check this scenario that represents what we’re hearing from our clients in the workshops:

“On a given project team the plan has about a 50/50 chance of being right, and the design is about 85% right, and the developers will get about 90% of the implementation right, and the testers have about 90% of the test cases right, and the release team has about 95% reliability of having the right stuff together to release, then your trust factor is something like

.5 x .85 x .9 x .9 x .95 = 32% Trust”   (and that’s much better than what we usually hear)

Ouch! That’s a 3x multiplier of the time it’s going to take no matter how fast we do all the delivery tasks.   In other words, we’re always limited by our trust factor. That multiplier comes into play with additional tasks we have to add into the equation to counterbalance the lack of trust. Then our equation becomes something like this:

T_DELIVERY = T_PLAN + T_RESCOPE + T_DESIGN+ T_{ARCH REVIEW} + T_DEVELOP + T_{TECH DEBT} + T_BUILD + T_REBUILD + T_DEPLOY + T_REWORK + T_TEST + T_RETEST + T_FIX+ T_REFIX + T_RELEASE + T_ROLLBACK + T_RE-RELEASE + T_EVALUATE

And that’s the issue our customers are facing today. There are a lot of wasteful tasks, checks, and balances that have crept into our delivery process due to lack of trust.

So, how does DevOps help this problem? It’s simple really; the DevOps approach is to attack the TRUST issue head on while simultaneously reduce the task time (T_x) in order to shorten the overall time to delivery. For example, if we apply a DevOps practice of breaking work up into small chunks, getting it out to users early, and getting the feedback, we can immediately impact the overall T_DELIVERY.That’s because we’re positively impacting the numerator and denominator with that practice. That is, by limiting the scope we make it easier to understand, less effort to transform to working code, test, and deploy, sooner to obtain feedback, and more likely to know if we’re on the right track.

Likewise for other practices!

The key is finding the balance of what practices to apply to maximize this equation for a given organization and their current bottlenecks & wastes.

DevOps practices address with three main impact points:

Speed to Value depends upon TRUST

TRUST is enabled by:
1. Clarity: A clear definition of desired outcome(s)
2. Collaboration: Team-wide communication and visibility
3. Consistency: Systematic and repeatable steps
When a team can master these three elements the time to perform each task will shrink and degree of TRUST will increase. DevOps practices impact these trust building elements:

Clarity:
- Break work up into smaller chunks and iterate
- Seek a minimum viable product (MVP)
- Define clear outcomes, ‘sponsor’ users, and playback (Design Thinking)
- Reduce dependencies with loosely coupled architectures (containers, micro services, SOA)
Collaboration:
- Form cross functional teams (end to end)
- Use big visual radiators
- Have a ‘single source of truth’ for all development assets – Requirements, Code, Deployable Assets, Infrastructure as Code, etc
- Be transparent with metrics
- Plan and reprioritize frequently
- Actively seek user feedback
Consistency
- Automate, automate, automate
- Continually improve by replacing manual tasks with automation
Speed to Value will be enabled by eliminating those bottlenecks and waste that have developed due to lack of trust. The key for organizations is to recognize that the current practices are maxed out and not sustainable when trying to move at higher speed. There’s a perfect storm amassing with big data, analytics, cloud, mobile and IOT that will either drive customers to make a DevOps transformation quickly or be a victim of the storm. Try using this simple math to help organizations understand how to move ahead and how our IBM solutions can be applied to attain speed to value.

This year I have had the great experience to work with our DevOps CTO Sanjeev Sharma, our customer thought leaders such as Carmen DeArdo of Nationwide, and our field teams around the world addressing DevOps transformations. Sanjeev, Carmen and I decided to summarize our experiences in a three part blog. This is the first of that three part series.

To learn more about IBM’s DevOps approach, check out www.ibm.com/devops. Also, watch the replay of a recent webcast on Variable Speed DevOps, I had a chance to present on, with Sanjeev Sharma and Carmen DeArdo.

About the Author/ Lee Reid

Lee Reid, Senior IT Specialist, IBM, is a senior solution architect with focus on practical adoption of software engineering tools, software delivery solutions, application life cycle management, and DevOps. He has over 25 years of experience in software engineering, design, programming, testing, development, tools & methodology, team leadership, and strategic planning.
June 22, 2015
7,600 Open Source Projects Per Company (and how it impacts DevOps)

That Supplier is Better For You

Since releasing the 2015 State of the Software Supply Chain Report, there has been a lot of great discussion across the industry on best practices for managing the complexity introduced by the volume and velocity of the components used across your software supply chain.

Today I want to focus on the huge ecosystem of open source projects (“suppliers”) that feed a steady stream of innovative components into our software supply chains. In the Java ecosystem alone, there are now over 108,000 suppliers of open source components. Across all component types available to developers (e.g., RubyGems, NuGet, npm, Bower, PyPI, etc.), estimates now reach over 650,000 suppliers of open source projects.

Source: ModuleCounts.com

However, like in traditional manufacturing, not all suppliers deliver parts of comparable quality and integrity. My latest research, the 2015 State of the Software Supply Chain Report, shows that some open source projects use restrictive licenses and vulnerable sub-components, while other projects are far more diligent at updating the overall quality of their components. Choosing the best and fewest suppliers can improve the quality and integrity of the applications we deliver to our customers.

While I am hosting a webinar next week to share many of the detailed report findings, I wanted to share a few of the more meaningful stats here.

Your 7,600 Suppliers

My research for the report revealed many new perspectives on “suppliers” across the software supply chains. First of all, I saw that the average large development organization consumed over 240,000 open source components last year — sourced from over 7,600 open source projects.

Average number of open source component downloads per organization in calendar 2014

On the surface, the huge reliance on open source projects is a great thing. Development teams have chosen to not write those pieces themselves, but have sourced the needed components from outside suppliers. This practice speeds development, enables more innovation, and ensures time-to-release goals are achieved. The use of open source is so prolific today, few of us could ever imagine reducing the use of those components and their suppliers in the future.

At the same time that we benefit from open source, our high paced, high volume consumption practices don’t allow us the time needed to do the due diligence on the suppliers or open source projects where we source our component parts from.

Average number of known vulnerabilities downloaded by organization in calendar 2014.

For example, of the 240,000 average component downloads in 2014, the same businesses sourced an average of 15,000 components that included known security vulnerabilities. In many cases, developers were downloading vulnerable component versions, when safer versions of those same components were available from the open source projects. While no one intends to download components with known vulnerabilities, the problem is exacerbated due to the lack the visibility into a better recommended version.

Fewer Suppliers, Less Context Switching

Choosing an open source project supplier should be considered an important strategic decision in organizations because changing a supplier (“open source project”) used is far more effort than swapping out a specific component. Like traditional suppliers, open source projects have good and bad practices impacting the overall quality of their component parts.

Traditional manufacturing supply chains intentionally select specific parts from approved suppliers. They also rely on formalized sourcing and procurement practices. This practice also focuses the organization on using the best and fewest suppliers — an effort that improves quality, reduces context switching, and also accelerates mean time to repair when defects are discovered. One industry example from the report describes how Toyota manages 125 suppliers for their Prius to help sustain competitive advantages over GM who manages over 800 suppliers for the Chevy Volt.

By contrast, development teams working with software supply chains often rely on an unchecked variety of supply, where each developer or development team can make their own sourcing and procurement decisions. The effort of managing over 7,600 suppliers introduces a drag on development and is contrary to their need to develop faster as part of agile, continuous delivery and devops practices.

Coming to Terms

When you come to terms with the volume of consumption and the massive ecosystem of suppliers you can source your components from, you quickly realize it is impossible to address this issue with a manual review process. Any organizations clutching to these outdated manual practices are will continue to be outgunned by the velocity by their software supply chains.

Just as traditional manufacturing supply chains have turned to automation, software development teams need to take the same approach by further automating their software supply chains. Information about suppliers and the quality of their projects needs to be made available to developers at the time they are selecting components. Information about the latest versions, features, licenses, known vulnerabilities, popularity of versions being used, and the cadence of new releases should be made available to developers in an automated way. Automating the availability of this information about suppliers can lead to better and fewer suppliers being used.

Be sure to read the full 2015 State of the Software Supply Chain Report for more information about open source suppliers and organizations sourcing practices. The report also highlights current and best practices being used in organizations that are managing their use of suppliers that feed their software supply chains.

To hear more about the overall report findings and industry best practices, please join me on Wednesday, June 24th (1pm ET) for our webinar.

June 19, 2015
DevOps Leadership Series: Gov Does DevOps (Part 2)

During my second day at DevOpsDays DC, I had the opportunity to catch up with a couple more industry thought leaders. First up, John Willis, (@botchagalupe) DevOps Days core organizer and co-author the upcoming “DevOps Cookbook”.

In this episode of DevOps Leadership Series, John highlights his first introduction to DevOps and the journey it has taken him on, and what about the current state of DevOps makes him excited. He says DevOps is really happening everywhere. Today large organizations and CIO’s of government agencies are now experimenting with DevOps at all stages. He references legacy companies that do not necessarily have “aggressive IT”, and how they now want to share their DevOps experiences.

https://www.youtube.com/watch?v=jU1O2bCU_hc

After that, I had the chance to chat with Leon Fayer, (@papa_fire) Vice President at OmniTI, following his DevOps Days DC presentation. Leon illustrates the things he found most interesting from the DevOpsDays Conference. He noted that a lot of people have started to pay attention to the monetary strategies, and how to monitor from top-to-bottom in a DevOps fashion. Leon was happy to see that people are now more generally interested in how to generate and collect data more accurately to solve their problems.

https://www.youtube.com/watch?v=kx4Y3P3Njro

June 17, 2015