devopsy.kinsta.cloud

Category: Blogs

Eight steps to a sustainable DevOps transformation
DevOps is not a standard. It is more a philosophy described by a loose set of concepts and tools, many rooted in the agile methodology and continuous delivery. That often causes confusion about the potential scope of DevOps, and makes it difficult to determine a definitive start and end point for a DevOps initiative. Moreover, DevOps requires significant cultural and organizational change, which can be hard to achieve and sustain.

So now we know what DevOps is, here we propose eight steps to a sustainable DevOps transformation, based on the use of a modified version of the Kotter change framework (1):
1. Develop a sense of urgency
“Deliver it faster and less expensively”. The two most common requests from the business—whether it is to deliver digital transformation, find more agile and cost-effective alternatives to internal IT, or other requirements.

IT organizations must become fully aware of the urgency to change in order to stay relevant for the business.

In the context of dev/ops this urgency is measured by release velocity, turnaround time for new features and other metrics. Benchmarking helps here: comparing delivery time results with other equivalent companies using DevOps tools, for example. Any deficiencies can be brought up through a value stream analysis with a TIMWOOD waste analysis.
1. Form a guiding coalition
For successful DevOps transformation, identify a team which is both supportive of the change and able to communicate progress. Include senior IT management from operations and development, plus key stakeholders in DevOps processes.
1. Create a vision
The DevOps vision for continuous delivery should be aligned with the business vision. It should be broad enough to cover all aspects of DevOps, but sufficiently narrow to guide decisions in the subsequent change process.

Gartner use the term “boundary objects” to denote things that can be “interpreted differently across different communities, but that posses enough common understanding to make them useful mechanisms to communicate with and improve knowledge”.[1] Boundary objects are thus considered a possible way to align on major concepts; the objects are flexible to local requirements, yet robust enough to maintain a common identity across sites. A DevOps/continuous deployment roadmap is also a necessity, encapsulating important themes and their priorities on an approximate timeline. Avoid detailed project plans, on the basis that DevOps should follow an agile approach and is all about continuous improvement.

Should the whole organization adopt DevOps or should DevOps remain limited to those areas in which the need for agility ultimately beats strong process compliance needs, risk mitigation and availability requirements?

There is no definite answer to this, but keep in mind:
- DevOps is a lot about culture. In the long-term it will be difficult to maintain a single organization, built upon two different cultures.
- IT systems are often inter-dependent; simple changes have to be implemented across an array of systems.
1. Communicate the vision
Communicate the vision, strategy and roadmap again and again. Moreover, start to live the cultural values of continuous delivery, such as avoiding blame, putting individuals before the process and promoting transparency.
1. Empower others to act on the vision
Many aspects of the vision—such as reduced process complexity and a revised understanding of collaboration—should already diffuse into the company culture even if “full” DevOps is not in place yet.

The guiding coalition needs to remove obstacles (such as too many process steps or approvals and rigid system access rights). These potentially limit the ability to experiment with new tools or methods. Indeed, any systems or structures that seriously undermine the vision should be replaced. Likewise, risk taking and nontraditional ideas, activities, and actions should be encouraged.
1. Create short-term wins
For successful DevOps deployment and transformation it’s important to deliver visible performance improvements, preferably a couple of pilot projects using established DevOps teams.

When choosing the pilot team, ensure:
- The team can be trusted to manage continuous delivery
- The application is a good fit in terms of architecture and technology
- The DevOps benefits are apparent to the business
1. Consolidate improvements and scale DevOps
Use the credibility derived from successful pilots to change systems, structures and policies that don’t fit the continuous delivery vision.

Have the different teams implement their own “flavors” of DevOps, best suited to their specific environment.

These teams can leverage the experience, processes and tools developed through the pilot, but are not required to do so as long as their implementation fits into the overall continuous delivery vision.

Avoid governance frameworks, which run contrary to the concept of agile.
1. Institutionalize new approaches
Articulate the connections between the new behaviors and corporate success with continuous delivery. To do this, you need the right metrics in place, linked to top-level KPIs. At this stage the entire business system has to be adopted to the new culture so that the change becomes part of “business as usual”.

This “institutionalization” also needs to be supported by metrics, incentive schemes and the appropriate style of management.

(1) John P. Kotter, The Heart of Change, 2002

[1] Gartner, How to Scale DevOps Beyond the Pilot Stage, Cameron Haight, 11 March 2015, https://www.gartner.com/doc/3003922/scale-devops-pilot-stage
June 29, 2015
Visualizing and defining requirements comes to DevOps

We have all seen the nice circular diagrams of a DevOps methodology for application lifecycle management (ALM) like the one above. We interject some automation, make the two halves work closer together, get some feedback loops going and go as fast as we can. But lets look at those first two steps in the diagram, the ones labeled consult and design. What and how do we accomplish this? There is a company called iRise that has built a great business by allowing for rapidly visualizing and defining requirements for building applications. It is high time we recognize this as the important piece of the puzzle that it is.

I recently had a chance to meet with some of the executive team of iRise while I was in London for the CD Summit. The premise of iRise is so simple I was thinking to myself, “why hasn’t anyone else thought of this?” If the old adage of crap in is crap out is true, why wouldn’t we take the time to define and plan our software better early on? I know everyone thinks they do, but do they really? Do we quickly visualize to help define the requirements and to help the team buy in and sign off? If it is set up right to begin with, it flows a lot faster later.

Now the folks at Tasktop creators of the Eclipse Mylyn open source tool have teamed with iRise to OEM iRise into the Tasktop Software Lifecycle Integration system. In speaking with Stephen Brickley, EVP of iRise, he thinks this partnership will be a key factor in bringing iRise into the mainstream DevOps tools market.

iRise’s customers, like software departments everywhere are adopting new ways of developing and operating software. New times and new ways don’t mean that you throw out what works though. To me it is a no brainer. If you have a tool that lets you quickly visualize, prototype and communicate to the rest of the team before investing a lot of effort into development, it really has to help. It is the ultimate shift left.

I guess the issue is how quickly can you do this without adding extra steps to the development process. From what I saw of iRise it makes it really easy and fast. Of course I would be interested to hear from anyone who has used iRise or even a similar tool on your own experience. But if it all is as it seems, I would expect iRise to be a welcome addition to the DevOps toolbox.

What do you think? What tool do you use for this now or do you just skip this in planning and initial requirements? We are always on the lookout for new tools to report on at DevOps.com, but ultimately we depend on our readers to tell us what is really useful or not. So let us know.

June 26, 2015
Open source and Samsung take center stage at Red Hat Summit

While much of the DevOps world was focused on DockerCon in San Francisco this week, there was also big news happening on the east coast. While DockerCon was shaking things up with the announcement of the Open Container Project the Red Hat Summit in Boston held its own share of big news and much of it spotlighted the important role of open source.

“Red Hat is really the independent voice of Linux focused entirely on business,” said Rob Enderle, principal analyst with the Enderle Group. “It is also a showcase of how to make money on the platform without taking advantage of anyone in the process and largely remaining true to the core tenets that created Linux in the first place.”

Focus on Open Source

One of the primary core tenets—if not the defining core tenet—of Linux is open source. The shared development and collaborative nature of the operating system as an open source project is its main strength. At the Red Hat Summit open source played an even larger role than normal.

“The company continues to show strong momentum for OpenShift Commons as the re-write of the technology is moving forward showing that the platform-as-a-service (PaaS) battles are alive and well,” explained Al Hilwa, program director of software development research for IDC. “Red Hat also rebranded and updated its FeedHenry platform as it continues to turn it fully into open source and integrate it with OpenShift.”

Samsung Partnership

Aside from the focus on open source, the other big news out of the Red Hat Summit was a new strategic alliance with Samsung. Samsung will push Red Hat’s mobile platform and potentially other software into the enterprise in a more integrated fashion as a function of the new partnership.

The relationship has potential, but according to Enderle there are both positive and negative outcomes possible. “Samsung needs Red Hat far more than Red Hat needs Samsung because Red Hat provided the enterprise brand that Samsung needs to penetrate the space. Samsung is a firm that has never found a platform they didn’t like and have historically been unable to focus on any one of them for extended periods of time,” cautions Enderle. “Red Hat alone won’t be able to fix that and Samsung’s behavior could actually tarnish Red Hat if they aren’t careful.”

Strong Momentum

Hilwa also made another observation about the strength of Red Hat’s business model. “That the company will cruise through the two billion dollar annual revenue barrier sometime this year is all but certain with its posted rates of growth. This is an enormous milestone for the open source world.”

We’ll have to wait and see how the Red Hat / Samsung partnership pans out for either company, but the Red Hat Summit highlighted the strength of Red Hat’s momentum and the unstoppable juggernaut of open source development.

June 25, 2015
Rancher Labs hopes to help DevOps teams rope in container management

DevOps teams have turned to containerization in a big way in order to help improve their efficiency. But many enterprises have run into problems scaling containers across their infrastructure to account for issues like networking and load balancing.

Rancher Labs, a Cupertino-based cloud software startup, is hoping to help organizations address the scalability with a new open-source Docker platform. Eponymously dubbed Rancher, the beta version for the platform went live this week. It’s designed to be a full set of tools to help DevOps teams manage Docker containers in production. The platform folds in infrastructure services built specifically for containers, including networking, storage management, load balancing, service discovery, monitoring, and resource management.

The goal of containers is efficiency and full portability, but Sheng Liang, co-founder and CEO at Rancher Labs, notes that the implementation can be hard to get perfect.

“As users deploy containers across different infrastructures, they quickly realize that different clouds, virtualization platforms and bare metal servers have dramatically different infrastructure capabilities,” said Liang. It’s these variables that Rancher seeks to limit. “By building a common infrastructure backplane across any resource, Rancher implements an entirely new approach to hybrid cloud computing,” said Liang.

The company is going long on containers—an assessment that backers agree with. The company also recently announced that it has received $10 million in series A funding from Mayfield and Nexus Venture Partners. In addition to its Rancher suite, the company also produces RancherOS, a minimalist Linux distribution tightly focused on running Docker containers.

The management team behind Rancher is one that is familiar to many cloud practitioners. Co-founder and CEO Sheng Liang was co-founder and CEO of Cloud.com, and continued on at Citrix as CTO of Cloud Platforms Group after the company’s 2011 acquisition. Also on board are Cloud.com and Citrix veterans Shannon Williams as vice president of sales and marketing, Darren Shepherd as chief architect and Will Chan as vice president of engineering.

According to industry analyst Paul Burns, Rancher Labs’ niche is ripe for new innovation.

“In its current state, running any containerization solution at scale and in production requires a significant amount of effort from those adopting the solution,” says Burns, who is president of analyst firm Neovise. “Key functions such as networking, service discovery, storage and load balancing can cause integration issues, keeping enterprises from truly reaping the benefits desired. By providing a unique solution that eases and simplifies the management of container software, Rancher is allowing companies to better utilize and benefit from the technology.”

June 25, 2015
Do/Should Developers Own Infrastructure?

In the pre-DevOps era, there wasn’t much question of developers owning infrastructure, if only because for the most part, they didn’t want to own it. They wanted infrastructure to work for them (or at least not get in the way), but they didn’t want to be stuck with the time-consuming and generally thankless task of maintaining and managing it. Add to that senior management’s understandable desire to keep infrastructure in the hands of trained IT personnel, and the IT staff’s equally understandable desire to maintain an orderly network, and the result was a relationship between developers and infrastructure that was cool and somewhat distant.

But with DevOps, we are entering (or have already entered) the era of infrastructure as code, and with it, the developer-infrastructure relationship has changed. If infrastructure is code, then almost by definition, infrastructure becomes to some degree a function of development, or at least so hard to separate from development that the distinction becomes almost irrelevant.

(more…)

June 25, 2015
How ITIL Can Co-Exist With DevOps
The original ITIL framework definition was published in the 1980s, but contrary to many opinions, it has evolved since then, provided one takes a more flexible view. Fast forward to ITILv3, and the framework has been shifted to the view that IT is a service that needs to support business initiatives and strategy.

In parallel, the “DevOps Movement” has continued to gain momentum, and is now the “culture” at many forward-thinking organizations. So, that begs the question, “Can I have my ITIL framework and still transform my IT Org to embrace DevOps?”

First, as a refresher, let me quickly outline the key components of ITILv3.
1. Service Strategy — The service lifecycle must stay focused on meeting the business goals and objectives.
2. Service Design — Architectural blueprint for the service
3. Service Transition — Outlines the change management and release cycle practices
4. Service Operation — Focuses on the delivery and control processes for the service
5. Service Improvement — Focuses on how to continually improve the processes for Service Delivery
Now let’s lay out an engagement model for the transition of the culture to DevOps.
1. DevOps Strategy — Establish metrics that outline Business Objectives and Value
2. DevOps Design — Establish Capacity, Availability, and Security Best Practices
3. DevOps Transition — Establish Organizational Structure and Define SLAs
4. DevOps Operation — Shift to Agile; Programmatic CMDB and Change
5. DevOps Improvement — Automate Incident, Problem, and Event remediation; Feedback loop
As you can see, there isn’t a huge gap between the two worlds, and I assert that the main changes are cultural, not technical, but one can use technology to start shifting the culture.

This is where the concept of a Security Fabric plays an integral role. Here is an example of how an Enterprise Reference Architecture can be organized to combine the best parts of ITIL and DevOps:
1. Policy Definition — Allows you to define Users, Groups, Hosts, Layers, and Interdependencies
2. Secrets Management — Allows you to store Secrets in a vault and access them programmatically in a secure manner with a complete Audit Log
3. SSH Management — Allows you to securely manage SSH public-keys and session interaction with a complete Audit Log
4. Host Factory — Allows you to instantiate and register new hosts or containers into your infrastructure
5. Identity/Service-Defined Authorization– Allows you to control interaction between your hosts or containers via Policy and Gatekeeper
IT now has a powerful platform to map Business Objectives to Identity, Hosts, and Processes. Once that mapping is established, previously static and brittle activities such as managing Secrets and SSH public-keys, can be moved into the Continuous Delivery pipeline, which maps directly improving Service Transition, Operation, and Continual Improvement.

Agility and Scale are programmatically added via various tools, which allows for frictionless and auditable Change requests, as well as rapid response to Incidents, Events, and Problems.

The diagram below attempts to pictorially show the co-existence of ITIL and DevOps and how the features of a Security Fabric are common between the two.

The shift to a DevOps culture should be approached as a continual evolution. ITIL was a great starting framework, but over time added too much structure and rigidity. You can leverage the Best Practices of DevOps to overcome that and transform into an extremely agile and productive environment.

About the Author/Mike Kail

Mike Kail was Yahoo’s CIO and SVP of Infrastructure, where he led the IT and Data Center functions for the company. He has more than 24 years of IT Operations experience with a focus on highly scalable architectures, prior to joining Yahoo. Most recently, Kail served as VP of IT Operations at Netflix. Prior to that, he was VP of IT Operations at Attensity, where he was responsible for the Americas data center operations team; including managing various big data systems with their Hadoop cluster, HBase, and MongoDB components. He has been recognized widely for his insightful industry commentary on Twitter, and was recently named by the Huffington Post as one of the “Top 100 Most Social CIOs on Twitter.” He holds a B.S. in Computer Science from Iowa State University. Reach out to him on Twitter or LinkedIn.
June 24, 2015
Rework is Choking Software

Rework is Hell

“Software may be eating the world, but rework is choking software”, tweeted John Jeremiah (@j_jeremiah). To shed more light on what is choking software, new data was released last week in the 2015 State of the Software Supply Chain Report.

In its discussion of application quality and integrity, the report revealed that the average application includes 106 open source components. It is clear that the use of these components has benefited development tremendously in helping to speed time to market and improve innovation. While the benefits are undeniable, development teams are also delivering applications that are “insecure by design”. Of the 106 components per application, the report’s analysis revealed an average of 24 (i.e., 23%) have known critical or severe security vulnerabilities. Those same apps also showed an average of 9 restrictive license types (e.g., GPL, AGPL, LGPL).

By electively sourcing components with known vulnerabilities and potential license risks, software development teams are building up higher levels of technical and security debt. And in order to reduce some of that debt, unplanned and unscheduled rework is often required.

No Developer Intentionally Uses Bad Parts

I have never met a developer that intentionally wanted to use “bad” components in their applications. That said, I have also never met a developer that wanted to spend four hours researching the latest versions, open source licenses and known security vulnerabilities for components (including dependencies) they wanted to use in their applications. While 57% of development organizations have internal policies in place that direct “thou shall not use components with poor quality and integrity”, many have stated that the policies are not followed or are simply not enforced.

The problem is one of volume and velocity, while people clutch to old processes that simply can’t keep pace. The analysis in the 2015 State of the Software Supply Chain report revealed that billions of open source downloads occurred in 2014, of which 6.2% had known vulnerabilities (1 in 16 downloads). For some large companies, the volume of downloads exceeded 240,000 — where 15,000 (7.5%) had known security vulnerabilities. At these volumes, manual processes and paper-based policies to prevent poor quality or risky components from getting into your software have no possibility of keeping pace. While software development practices aim for higher velocity, through component-based development as well as Agile, continuous and DevOps practices, the processes to keep up with it have not changed enough.

The only way to keep pace with this velocity is through automation. And if we want to avoid unplanned rework associated with remediating known security vulnerabilities or selecting alternative components with approved license types, automation has to become the norm rather than the exception within development.

At Speed

“If I look at the mass, I will never act. If I look at the one, I will.” — Mother Teresa

Looking at the massive volume and velocity of open source and other artifact consumption can be discouraging and overwhelming. However, the volume and velocity within our software supply chains will not diminish—and without a new approach, the volume of unchecked quality and integrity of parts being consumed will continue to build up as technical debt.

Automation in areas of testing, build, and deployment has provided significant performance benefits. Likewise, investments in software supply chain automation have shown markedly improved efficiency and controlled risk, as the best practices in the 2015 Report illustrate. Automation can unleash the potential of an organization’s development capacity. Rarely is there such an opportunity to simultaneously increase speed, efficiency, and quality.

In order to improve the quality and integrity of components used in our applications, we need to release our clutch on old practices. We can further embrace automation across our software supply chains — and improve the quality of our applications — by considering these three steps:

1. Start with creating a software bill of materials for one application. Visibility into one application can help you better understand your current component usage. A number of free and paid services are available to help you create a software bill of materials within a few minutes. The bill of materials will help you to identify the unique component parts used within your application and the suppliers who contributed them. These reports list all components used, and several services also identify component age, popularity, version numbers, licenses, and known vulnerabilities.

2. Design approval processes to be frictionless, scalable, and automated. Manual reviews of components and suppliers cannot keep pace with the current volume and velocity of consumption. Your organization must not only define your policies for supplier and part selection but also find practical ways to enforce them without slowing down development or inadvertently encouraging workarounds. Policies must be agile enough to keep pace with modern development. Strive to automate policy enforcement and minimize drag on developers.

3. Enable developer decision support. Developers are primary consumers of components in software supply chains. They initiate every component request. Help developers by automating the availability of information on component versions, age, popularity, licenses, and known vulnerabilities within their existing development tools so it is easy to pick the best components from the start. By selecting the highest-quality components from the highest-quality suppliers early, you will improve developer productivity and reduce costs.

Here is the complete 2015 State of the Software Supply Chain Report. I will also be discussing more of the Report findings and best practices on a webinar scheduled for Wednesday, June 24, 12pm ET (register or view it on-demand).

The previous blog in this series was, Fewer and Better Suppliers. Next up, I will be reviewing Visibility and Traceability practices across the software supply chain with more commentary from the 2015 State of the Software Supply Chain Report.

June 24, 2015
Bimodal IT and Remodeling Traditional IT for Greater Agility
Bimodal IT is a Gartner concept created to help CIO’s understand that IT organizations need to support both traditional and agile modes of IT solution delivery and operation in order to make wise choices about infrastructure, process, people, and tools. Much attention has been given and rightfully so to the disruptive and innovative “Mode 2” teams building on the practices established by web-scale giants like Amazon, and Google. However, in enterprises there still exists a sprawling majority of “Mode 1”, traditional, industrialized applications run by teams that must focus on risk aversion, security and compliance. While these will likely not match the innovative velocity of agile IT teams and applications, traditional IT teams are still under pressure to get faster, more agile, innovative, productive and cost-efficient. Yet, the Mode 2 playbook can’t be applied wholesale to Mode 1 transformation work because it often assumes a greenfield situation—kind of like a new home construction. By contrast, Mode 1 modernization is more like renovating an existing home that must proceed in steps that address the present realities of applications, legacy infrastructure, business mandates, organizational interactions, and technical skills.

Understanding Bimodal IT

The Bimodal IT concepts is meant to distinguish between applications and teams that are traditional in their focus, and those that are agile. In her blog post on Bimodal IT, Gartner analyst Lydia Leong explains Bimodal IT thusly:

“Traditional IT is focused on “doing IT right”, with a strong emphasis on efficiency and safety, approval-based governance and price-for-performance. Agile IT is focused on “doing IT fast”, supporting prototyping and iterative development, rapid delivery, continuous and process-based governance, and value to the business (being business-centric and close to the customer).”

The reason that Gartner promotes the idea of Bimodal IT to CIO’s is to help them see that it can be dangerous trying to optimize processes, human resources, infrastructure, and tools to cover such different modes, and as a result, optimizing to neither. Rather, the right move according Gartner is to allow agile teams room to develop a brand new set of capabilities that can then help transform the whole organization. Lydia continues in her blog:

“We’ve found that organizations are most successful when they have two modes of IT — with different people, processes, and tools supporting each. You can make traditional IT more agile — but you cannot simply add a little agility to it to get full-on agile IT. Rather, that requires fundamental transformation. At some point in time, the agile IT mode becomes strategic and begins to modernize and transform the rest of IT, but it’s actually good to allow the agile-mode team to discover transformative new approaches without being burdened by the existing legacy.”

The Contrast Between Agile and Traditional IT

Traditional IT and Agile IT are different in a number of ways. Perhaps an effective way to explain the difference is to picture what these applications and teams look like.

The classic agile IT initiative in the enterprise is around a mobile, e-commerce or web-centric application that is aimed at moving into new markets, improving engagement with new customer segments, or disrupting a current market. The application is fully virtualized, and the preferred infrastructure is a public cloud such as AWS. The team is conceived as a unified, DevOps team of line of business product managers, developers, testers, and operations folks working collaboratively on an agile development methodology. Automation is built deeply into the process, with infrastructure as code tools used to manage public cloud virtual machines and related infrastructure components via well-documented RESTful API’s, plus other DevOps toolchain elements automating and managing most processes, including continuous integration, continuous delivery and continuous deployment of code into production.

Traditional IT stands in pretty stark contrast. Let’s consider an industrialized, mission-critical application that has evolved slowly for years. The application (which is usually composed of anywhere from 5 to 10 or more components) is likely running on a mix of legacy (think RISC UNIX), dedicated (x86-based) and virtualized servers (VMWare and perhaps Hyper-V) running in a private or hosted private datacenter, using a mix of traditional and software-defined storage and connected by Ethernet and Fibrechannel switches. The application components themselves may be legacy code, making them difficult to quickly port to more modern infrastructure. For example, I was talking recently with a colleague who told me about his sister who still writes code in Fortran for a major bank. With such a large, complex, long-lived and mission-critical application suite, the IT organization maintaining it and the developers creating updates and upgrades are highly concerned with quality, reliability and predictable performance. The nature of the overall team working on an industrialized application is a far cry from the sort of “two pizza” DevOps teams on the agile IT side. Rather, it’s more of a loose consortium of teams such as developers, testers, security and compliance, plus outsourced contractors and vendors. The development process is waterfall-based, managed by an application lifecycle management (ALM) suite of some sort. Automation is scant, and in particular it takes a long time to pull together infrastructure for those dev, test and other teams to perform their jobs. Despite that, a production-like environment for testing, security, compliance, etc. is considered necessary to minimize downtime and other quality risks. As a result, release cycles can stretch into many months.

Modernizing Traditional IT

When my wife and I remodeled our 1908 Craftsman-style house about 15 years ago, we discovered that an earlier renovation had actually cut out a third of a key supporting beam, leaving one part of the house literally hanging on a beam that was only attached to one side of the house. Needless to say, we were a bit alarmed to find that out. When dealing with mission-critical applications and infrastructure, you can’t be careless. As mentioned earlier, simply taking the playbook for a greenfield, agile IT application and trying to apply wholesale it to a legacy, industrialized application usually won’t work. Making traditional IT more agile requires a step-wise renovation—a risk-mitigated path that leads towards agile and DevOps practices, but without a great leap into the dark. The steps need to establish the building blocks to agile/DevOps practice without disrupting the application, without introducing downtime, and while delivering solid returns along the way. Successful legacy application and service ‘remodeling’ efforts observed in highly traditional organizations such as telecommunications and cable operators provides good ideas on how to take those steps:
- Identify or Procure Automation Skills: The initial steps to remodeling Traditional IT are about incrementally improving current practice with automation and at the same time making it ready for agile and DevOps. This means that to be successful, you need to get some automation skills into the mix. A team that can’t take hold of its own destiny in modernizing a Traditional IT application is not going to have a sustainable change, and that means identifying or procuring some automation expertise and protecting that person’s time to focus on the renovation rather than on firefighting.
- Private Cloud IaaS: One of the most serious obstacles to agility is the painful amount of time and effort it takes to allocate and provision infrastructure for developers, testers, etc. and the lack of standardized, consistent infrastructure environments, due to manual processes. This is exactly the issue that drives some development teams to go shadow IT and start developing on AWS. Often times, these development teams then discover that integrating that software into the production environment is plagued with translation issues. Nonetheless, the first order of business then is for the infrastructure team to establish IaaS that can serve up full production-like infrastructure environments to devs, testers and other constituents. This isn’t trivial when you’re dealing with a combination of legacy servers, bare metal x86, VMs, and perhaps networking, security and other devices.
The IaaS delivers a number of key improvements. First of all, it speeds up access because developers, security, compliance and other personnel can now get access to infrastructure in the space of minutes rather than the space of days or weeks. More than just speed, IaaS enables asynchronous productivity by various teams to happen 24×7, so that quality progresses in an even fashion, unimpeded by office hours. This is especially helpful if there are global teams, contractors or vendors involved. IaaS also provides standardization and consistency across all efforts. Finally, IaaS saves costs. Significant amounts of operations personnel time that was devoted to painstaking manual infrastructure assembly can be repurposed to higher value activities, and infrastructure utilization rates can easily double or triple almost immediately, which reduces the need to purchase more capital equipment.

A word here about GUI’s. One of the key notes of dissonance that often occurs is when people hear developers say that they don’t want GUI’s, they want API’s. That in my view is absolutely valid, but more so for Mode 2 Agile IT teams. It turns out that IaaS GUI portals can be rather helpful in providing an easily understood reference point to multiple teams that may have highly varying understandings of the end-to-end picture of the application and its infrastructure. For example, if an IaaS self-service portal provides a full visualization of the end-to-end infrastructure involved, a developer who might assume that latency is always near zero might become more aware that there is a WAN in the middle of the end-to-end infrastructure.
- Test as a Service: Many development and testing teams have taken some stab at test automation. However, test automation tends to live and be maintained at the individual level. The natural follow-on to IaaS is to build a standardized regression or certification of automated tests that can be offered from the IaaS portal—test as a service (TaaS). This allows teams to access production like environments, do their development work, then easily launch a comprehensive certification test. Tests naturally should report their results into the ALM tool so that individuals and teams can review results in near real-time to keep the process moving. The benefits are similar to IaaS: increased velocity, standardization, productivity and efficiency.
- DevOps Evolution: With a combination of IaaS and TaaS, the stage is now set to deploy a DevOps toolchain and start doing cross-functional technical collaboration, and agile methodologies. For example, the IaaS and TaaS can be launched by a unified continuous integration server upon software development check-ins from any team. Of course, DevOps doesn’t just mean tools and automation, but the process of coming around collaborative devtest infrastructure and common certification tests to build a common, agile development practice is one of the steps towards a more broadly responsive cycle.
Conclusion:

Traditional IT may not be the sexiest beast in the technology forest, but it’s often mission-critical. Gartner’s Bimodal IT concept encourages CIO’s to let go and let Mode 2 Agile IT teams do their thing to create new culture and practices. Translating that to Traditional IT requires a more step-wise approach. Progressing from manual processes to IaaS to systematized, self-service test automation as the foundation for DevOps evolution is a sound way to proceed. With these steps. Traditional IT can “do IT right” and do IT faster.

About the Author/ Alex Henthorn-Iwane

Alex Henthorn-Iwane joined QualiSystems in February 2013 and is responsible for worldwide marketing and public relations. Prior to joining QualiSystems, Alex was vice president of marketing at Packet Design, Inc., a provider of network management software, and has 20+ years of experience in senior management, marketing, and technical roles at networking and security startups.

Through his roles at QualiSystems, Packet Design, CoSine Communications, Corona Networks and Lucent Technologies he has acquired expertise in cloud computing and the opportunities presented through virtualization. He has written for Embedded Computing, Virtual Strategy Magazine, Datamation, SDN Central, Datacenter Knowledge and InformationWeek.

Connect with him on LinkedIn, Twitter, or SlideShare.
June 24, 2015
OCP is a victory for CoreOS and Containers in general

A broad coalition of major tech companies have joined forces to create the Open Container Project—or OCP. The move to develop and maintain common standards around software containers vindicates the CoreOS split from Docker and reunites the two in a common cause that will make all container platforms and technologies stronger.

A press release announcing the Open Container Project explains the overall vision for OCP. “Housed under the Linux Foundation, the OCP’s mission is to enable users and companies to continue to innovate and develop container-based solutions, with confidence that their pre-existing development efforts will be protected and without industry fragmentation.”

Docker and CoreOS are at the heart of OCP but they’re by no means alone. A veritable “Who’s Who” of DevOps and container technologies has enlisted to support OCP, including Amazon Web Services, Cisco, EMC, Google, HP, IBM, Linux Foundation, Microsoft, Red Hat, and VMware.

At one point Docker had a Standard Container Manifesto that spelled out some basic attributes it felt all containers should have. The Standard Container Manifesto called for containers to operate in a standard fashion, be content agnostic, infrastructure agnostic, and designed for automation among other things. When Docker removed the manifesto and seemed to shift to a more proprietary strategy CoreOS split off and launched its own container technology to continue the vision of a container standard.

“An open, well-defined specification is required for the overall success and adoption of containers. At CoreOS, when we started the App Container spec (appc) our goal was to have a well-designed software container specification. With the OCP, we have now come together with Docker and many industry giants in an effort to unify this goal.”

The launch of OCP and the laundry list of companies backing it show that CoreOS was not alone in its pursuit of the vision for a container standard. “We are excited to support this effort to bring the industry together to agree on a core standard for containers, which we feel will benefit all customers no matter what cloud provider they use,” said Jason Zander, CVP, Microsoft Azure. “Through our close work with Docker over the last year we have demonstrated our commitment to openness and welcome this opportunity to help create an open platform for customers.”

“Customers need an open, interoperable future for software containers and we are happy to see the industry coming together to establish a common standard,” explained Kit Colbert, vice president and CTO, Cloud-Native Applications, VMware. “Our participation in the Open Container Project reinforces our goal to help customers innovate more rapidly and benefit from containers as they adopt the technology in their environments. We look forward to working with the community on container portability and security which are areas of focus for enterprise IT.”

According to the OCP press release Docker will donate the code for its software container format and runtime—as well as the associated specifications—to the project. The leadership team from the Application Container spec (appc) will also contribute technical leadership and support for OCP.

Ben Golub, CEO of Docker, summed up by acknowledging the effort and support for OCP. “We’d like to thank the many people in the industry who came together to support this important initiative. Our thanks to the Linux Foundation for the incredible work they have done in helping to pull this broad group together. Given the criticality of the intersection of containers and operating systems, we’d also like to thank Red Hat and Microsoft for their early leadership on this initiative. Our sincere appreciation to IBM and Intel for their ongoing encouragement of open governance. Finally, we’d like to recognize the members of the appc community, including Google and CoreOS, for their willingness to join all of us to constructively address these critical issues.”

Now that all of the major players are united in a common cause to define and maintain an industry standard for containers it will be much easier for developers and organizations to embrace container technologies without reservation. The confidence that your investment of time, money, and effort won’t be wasted or paint you into a corner is very liberating.

June 23, 2015
Three Ways Dev and Ops Teams Can Manage the Integration Lifecycle
Many enterprise organizations did not invest enough money into integration as they built their cloud or hybrid infrastructures. As if integrating on-premise systems and applications wasn’t tough enough, in today’s cloud era with the rise of hybrid environments, connectivity has only gotten more complicated – especially when combined with the increased speed of deployment. Puppet Labs found that high-performing organizations deploy code 30 times more frequently than their lower-performing counterparts, meaning dev and ops teams are struggling to break down data silos while simultaneously managing constant technology upgrades for line-of-business users.

There’s no doubt that, whether public or private or hybrid, the cloud is now part of most companies’ infrastructures, and its share will only increase. Goldman Sachs estimates that spending on cloud computing infrastructure and platforms is expected to grow at a 30 percent CAGR from 2013 through 2018 compared with 5 percent growth for overall enterprise IT, while Forrester predicts that 25 percent of all software applications purchased will be cloud-based by 2020.

With the speed and agility the cloud imposes on businesses, dev and ops need to come together to look at data integration throughout its entire lifecycle, from developing new apps or services on top of existing apps, all the way to the ongoing service, maintenance and upgrades of those applications. The two teams must join forces to manage this process as one comprehensive integration lifecycle, where dev and ops play equal roles solving for new data integration challenges brought about by hybrid deployments and siloes created by line of business users’ cloud purchasing habits.

Today’s dev and ops teams have to look at data integration as a service, not just a one-off implementation they can hand off. Here are three strategies to help bring the two teams together to jointly manage the data integration lifecycle.
1. Assess Current Integration Challenges Collaboratively
  Without alignment between dev and ops teams, integration lifecycle can’t be managed successfully. Organizations need to align on the people and process portion first, asking questions like: what systems does each business unit need data from? How likely is it that their needs will change, and how can you plan for a flexible integration as a service? What capabilities will ops need after dev implements the integration?
After reaching agreement, teams will need to jointly assess and select a data integration as a service technology or platform that allows for joint requirement setting, collaboration, testing and ongoing adjustments throughout the lifecycle of the applications. Selecting the right integration platform as a service at the onset is critical to minimize cost, time and wasted dev and ops cycles down the line as business and technology needs change. The most agile integration option typically consists of a plug-and-play, flexible platform that makes use of APIs and also allows for deeper customization down the road.

2.Understand the Four Key Stages of Integration Lifecycle Management
Dev and ops teams are now part of the same cycle dealing with frequent updates and agile product development. It’s no longer about developing, testing and then having ops maintaining, but about a seamless and transparent handoff from dev, to ops and back throughout the entire application and data integration lifecycle.

To capture the most value from disparate organizational data and to minimize ongoing data and application synchronization costs, dev and ops teams need to adopt a joint integration management strategy that accounts for the four distinct phases of data and application connectivity. These include:

3. Design phase. After understanding business use cases and selecting a platform, dev and ops must collaborate on priorities – what near-term functionality can they design for based on the plug-and-play layout, and how can they tackle that low-hanging fruit while maintaining the flexibility to meet future requirements?

By starting with the right technology and a common integration framework, dev and ops teams can design connected core systems and data early on with plug-and-play API connectivity for solutions like CRM and ERP, while maintaining the ability to easily expand functionality with custom/deeper integration capabilities with marketing automation, business intelligence and other apps.

In addition to functional capabilities, the design phase should address performance and scalability by taking into account throughput and latency. This process involves planning to meet performance expectations, even with the customization and extensions users will likely want down the road. Dev and ops should ask themselves questions like: which company systems will be on premise, in the cloud or hybrid? What are the performance, scalability and security tradeoffs of each approach?
1. Develop phase. Dev and ops need to work together in the development phase to carefully document key aspects of the integration while building connectivity for specific use cases. This documentation will be critical to the seamless handoff between the two departments, so they both need to be involved in the process. Strategic collaboration and thorough documentation can improve the efficacy of maintenance and debugging activities, help companies adjust to staffing changes and more easily transfer integrations to and from the two teams during the ongoing development and roll-out process.
2. Deploy phase. Given that you’ll likely deploy in a hybrid environment, make sure you test both on-premise and cloud integrations to ensure that data is flowing freely. To help test integrations and move them into a production environment, look into staging and production environment tools that streamline the process. Infrastructure costs should also be considered, as they may increase unexpectedly if you don’t watch them throughout the deploy phase.
3. Run phase. Continuously monitor to confirm that the data is flowing as designed. If dev and ops selected the right technology platform, the teams won’t have to worry about ongoing API management, because the platform takes care of it.
The right platform will also lower downtime and maintenance costs with a multi-tenant management environment that allows the expert for any given situation to dive into the integration backend or runtime. The platform should also provide ongoing diagnostic reports and proactive alerting to allow ops to home in on potential performance issues.
1. Build a Responsive IT model
  With dev and ops coming together to take advantage of the speed and agility offered by the cloud, the resulting just-in-time services for businesses are turning IT from a cost center into an innovation center. The data integration lifecycle is another result of the new cloud-shaped world, helping dev and ops teams create a more responsive IT model and allowing them to better meet business needs while driving innovation. While shared environments bring with them an additional element of risk, a responsive IT model is an important step towards both short-term and long-term success in the modern connected enterprise.
These three steps allow dev and ops teams to offer the value, speed, efficiency and opportunity that today’s enterprises require. By adding collaboration and agility at every stage of the integration process to plan, implement, track, debug and update continually, dev and ops teams can scale to respond to the data-driven needs of the business more precisely.

How have you overcome cloud and hybrid application and data connectivity challenges in your organization? Connect with me in the comments below, or on Twitter at @scribesoft.

About the Author/John Joseph

John Joseph, VP of marketing for Scribe Software, earned a master’s degree in business administration from the MIT Sloan School of Management, a master’s degree in electrical engineering from the University of Southern California and a bachelor’s degree in electrical engineering from MIT.
June 23, 2015