The role of Quality Assurance (QA) is creating a bit of a buzz these days, because even as teams and budgets are shrinking, the demand for better quality is increasing. The chilling repercussions of service disruptions caused by software flaws warrants interest even at the executive level: “Nobody’s madder than me about the fact that the website isn’t working as well as it should,” said President Obama when HealthCare.gov went down last year due to software defects.
Category: Blogs
-
Infrastructure Automation and API Taxes
That automation is one of the critical components of DevOps is apparent in numerous surveys and studies that focus on the benefits organizations have realized from implementing the approach across operational groups. Automation itself starts with (or should start with) a comprehensive API. That means infrastructure – network and application focused – needs to expose via an API the myriad configuration options it requires as well as optional features it offers.
It is through these APIs, after all, that integration between devices and systems and across operational groups occurs. The much vaunted SDN or cloud “ecosystem” is really a euphemism for pre-packaged and validated integration. Integration that occurs via an API.
These same APIs are available to everyone, of course, and lay a foundation to operationalize app deployments by automating the various provisioning and configuration tasks required to move an app from dev to production.
APIs allow for fine-grained automation and orchestration, with each and every knob, button and slider option available to be flipped, switched or turned. The more features and functions possible, the more buttons and knobs and sliders there are to manipulate, figuratively speaking.
Each and every one of those calls to do so needs to be carefully managed. Exceptions happen, errors occur, and any script or tool ought to carefully validate that each and every one of those calls has been successful. Which is going to take a lot of code. Code that itself will introduce the possibility of errors and require maintenance and reviews and all that goes along with good development practices. Code that will also impact the time required to execute. Given that most APIs are RESTful and require invocation via standard HTTP exchanges, each and every API call incurs time to send, time to execute, and time to receive a response. The overhead is basically an API tax; for every API call you invoke, you’re paying with time… and more time.
Martin Fowler touches on this in his post “Microservices and the First Law of Distributed Objects“:
The consequence of this difference is that your guidelines for APIs are different. In process calls can be fine-grained, if you want 100 product prices and availabilities, you can happily make 100 calls to your product price function and another 100 for the availabilities. But if that function is a remote call, you’re usually better off to batch all that into a single call that asks for all 100 prices and availabilities in one go.
Thus for sufficiently complex systems another option is needed: templates. Templates do not replace APIs, but rather provide an additional and far more compact method of accomplishing the same tasks automatically, but without the overhead imposed by highly granular sets of API calls. Templates encapsulate a configuration; codifying the settings for a variety of knobs, buttons and sliders into a single configuration artifact. In a fine-grained API-base integration, each of those settings must be maintained somewhere external to the automation script or system. Each API call will include one (or maybe two) of those variables as read from some file or system that maintains the application-specific set of data. This introduces many potential points of failure. Whether file or database, variables must be retrieved before being associated with an API call and then transmitted to the receiving device. Errors may occur on read, on format, or on transmission.
In contrast, if all the variables exist as a single, validated configuration template artifact, it can then be sent, as a single entity, via an API call to the device or service in question. The number of potential points of failure is effectively reduced to read/retrieve and on transmission. This greatly simplifies the process as well as reduces the time it takes to actually execute the configuration of the target device or service.
In addition to simplifying integration by reducing the provisioning and configuration tasks from potentially hundreds of API calls down to a small set of calls, this has the benefit of fitting much easier into an infrastructure as code strategy, where common configurations are treated as artifacts and stored in repositories much like its application counterparts.
There is an inflection point at which it becomes advantageous to use a template-based approach over individual API calls. That point occurs before the scripts driving the API-based automation become as complex as an application and thus bring with it associated issues like error rates, time spent troubleshooting and maintainability.
A template-based approach simplifies upstream workloads and integration as well as discrete automation scripts by reducing the set of variables required to just a few – usually those dependent on network-specific values such as IP addresses and network segmentation membership (tags or ids representing VLAN, VXLAN, NVGRE, etc…).
This further ensures a consistency (and in turn results in greater stability of the underlying infrastructure) in app deployments.
APIs are a good thing. They’re a key enabler of software-defined architectures like SDDC, cloud and SDN. But API-enabling infrastructure doesn’t necessarily mean only on a checkbox and radio-button basis. That can be valuable but it can also lead to integration efforts that are just as complex (or more so) than their manual counterparts. A template or policy-based (application-driven) approach coupled with an API through which to deliver and execute such constructs results in a much cleaner, more consistent and stable means of integrating provisioning processes into the greater software-defined architecture.
-
The Conundrum of Adopting DevOps within Organizational Constraints
I wish I had a dollar for each time I’ve now heard, “we’re adopting DevOps, but we have to operate within the constraints of what the business is ready to accept.” Basically, this is becoming the battle cry of many enterprise IT professionals that are DevOps believers, but are stuck with the limitations of operating within a complex, political and immovable business structure.
One on hand, it is very pragmatic to approach adoption of new methods, techniques, technologies, etc. on a small scale and then expand based upon success. However, what is the result of a DevOps initiative in which existing organizational constraints and bottlenecks continue to remain in place?
If you’ve been in the business world for more than a few years you’ve probably already had the experience of having a really good idea only to come to the realization that implementation of that idea requires cooperation of another organization. For example, Marketing might have a great concept for how to acquire new customers, but would require sales management to provide incentives to the sales team for it to succeed. For purposes of this example, let’s assume that marketing has made its case to sales and sales rejects the proposal (many times there is a perception that the idea will be rejected limiting those from even trying; this is a worst-case scenario for any business). In this case, Marketing might attempt to operate around the need for the sales incentive and, ultimately, come to the realization to abort the plan or lower expectations for results.
Is anything more debilitating to morale then end up with these two outcomes as the only choices? What would you do if you were in the position of driving this for Marketing?
Often times in businesses there is no grievance committee. To steal a line from “A League Of Their Own”, “there’s no crying in business!” What DevOps is surfacing is that business is about relationships and managing cross-departmental and extra-departmental relationships is not something that many IT professionals have had to or want to foster. Ultimately, this will have a detrimental impact on DevOps adoption.
In most cases, DevOps requires that some leader within IT steps up with the idea to foster a frictionless ideation-to-operation lifecycle. By frictionless I mean the removal of as many bottlenecks and constraints for transitioning from the idea phase to a production-ready support application. Like Marketing in my earlier example, this may require this leader to “sell” their idea to the heads of IT Operations, Application Development, Quality Assurance, Information Security, and potentially C-level executives. A “No” response by any of these teams will result in the leader working around this limitation resulting in no change for the processes and work output of that particular group or attempting to do what they can within the organizational boundaries within their control.
Hence, business agility is tied to velocity of accepting change, which is further dependent upon individuals fostering a strong enough personal relationship–at a minimum one of mutual respect–to be willing to accept risk in support of a larger goal. I believe many would see this as a high-risk constraint on the business agility deliverable.
One way to increase likelihood of avoiding this trap? DevOps needs to be sponsored from C-level management from the start inclusive of the necessary incentives required to help drive the changes that will lead to more rapid adoption of DevOps practices and speed the way to business agility. There’s enough evidence now available to convince most C-level executives that a DevOps program is a worthy investment in moving their IT organization toward greater alignment with their business goals, producing faster, reducing operating overhead and improving quality.
As with most changes within IT, and DevOps adoption is no different, there is a requirement for bottom up support as those closest to the work need to believe, and there is a top-down requirement that provides the coverage, financing. motivation and accountability to the business to allow the change to occur.
-
Closing the Gap Between Database Continuous Delivery and Code Continuous Delivery
The most important feedback I get about the current state of database delivery is directly from the DBAs I meet with. Their DevOps teams are stuck in manual database delivery purgatory. Development managers just want the database delivery to work like their software delivery already does. However, databases are a different beast altogether. Unfortunately, DevOps for Database has taken a back seat to source code, as we see the tools and methodologies perfected, leaving database automation to play a game of catch up.
Another issue is that DBAs don’t trust deployment automation like code developers do. All DBAs have been burned by breakdowns or conflicts at one point or another, and most of them just feel they need to take matters into their own hands to ensure the release is done right. We recently surveyed over 200 DBAs, and the largest percentage of them noted mistrust of automation as the biggest barrier to adopting continuous delivery for their databases. In the same survey, only half of the companies doing continuous delivery of their code said they are also doing DevOps for Database. This needs to change soon or database delivery will be stuck in the past and slow down all company releases.
Databases hold the most vital company information and they need to start being treated like they matter as much, if not more, that the code being written for a company’s software. So, the question is, how do we close that gap between source code automation and database automation?
The answer is simpler than you might think. We need to bring the same processes for source code continuous delivery to DevOps for the Database. What a novel concept!
DevOps for Database requires best practices just like source code. Deployment practices need to be enforced. Version control needs to be enforced. Safe automation processes need to be created. You need to include the database in the continuous process and feedback loops. The most important thing is to make sure you have a way to control these processes, and if something needs to be handled there will be automated notifications and red flags to point out the issue.
That being said, ensuring safe continuous delivery of databases is not so simple. Databases are not a collection of files; they hold client information, transaction data, application content, etc. To make automation really work there needs to be a solid transition code created to handle database schema structure, database code, and content such as metadata.
Furthermore, there are organizational-level changes that will be required in order to ensure proper continuous database delivery. Both DBAa and C-level management will be required to adopt these development methods for databases. It can’t be and isn’t a one-person job. Both sides need to advocate for database automation.
Everyone recognizes the importance of the database to the organization. That is the reason DBAs are so afraid to automate and management is not forcing it. It should be just the opposite! Their sentiments on database control and automation need to be more positive. The question they are asking is – what could go wrong if we bring continuous delivery to our database? The question should be – what could go wrong if we don’t bring continuous delivery to our database?
Yes, database deployment automation is not a simple process, but the fact is that ensuring continuous delivery means increased productivity, faster time to market, reduced risk for new releases, and higher quality with fewer bugs. Otherwise your database will keep lagging and eventually you can find yourself in a difficult situation down the road.
About the Author
Yaniv Yehuda, CTO, DBmaestro (http://www.dbmaestro.com/)
Yaniv is the Co-Founder and CTO of DBmaestro, the leading provider of DevOps for Database solutions which enable control of database development and deployment. Yaniv is also the Co-Founder and the head of development for the Extreme group, a leading IT services solutions provider group.
-
Transforming applications with Cloud and DevOps – one hybrid solution for multiple challenges
Are you on cloud yet or clouded? The answer lies in how you have adopted the transformation. Today, the world is Hybrid. For large enterprises and organizations, it always has been. Almost no enterprise is monolithic when it comes to their environments. They may already have different distributed environments and/or mainframes in their ecosystem. The advent of cloud has only added to this complexity and hybrid nature of the environments. Enterprises which have replaced their datacenter with cloud adoption, in some cases are unable to do so completely or even adopt a singular cloud solution as that remains the domain of born-on-the-web companies that started on the cloud (think Netflix) or started on physical servers and migrated very early to the cloud (think WhatsApp). You cannot think of a big bank or even a telecom giant that is a Cloud Service Provider (CSP) that fits this bill.
For the typical Enterprise (IBM included), the cloud has replaced and consolidated large sprawls of physical servers. However, physical environments remain, and may never go away and cost of migration plays a very important part in this retention. There may be large data-heavy applications that are hosted on distributed environments that may be just too expensive to re-host on the cloud. Then there are mainframe-hosted applications that cannot be migrated to the Cloud. Or the migration to cloud may be a massive multi-year transformation that you are only starting on and will have to live with workloads spread between the cloud and the physical world for an extended period of time.
What’s interesting is that, even the cloud itself will be ‘Hybrid’ in nature. There may be combination of on-premises and off-premises clouds adopted, with off-premise’s ‘cheaper’ cloud used for less critical workloads like Dev-Test environments, and on-premise’s cloud for business-critical workloads. There may also be the presence of a Platform-as-a-Service (PaaS) being used for developing innovative Systems of Engagement applications or for experimentation, which co-exists with an Infrastructure-as-a-Service (IaaS) Cloud for Systems of Record applications and other traditional workloads.
That is the complex world we live in.
DevOps and Hybrid Cloud
Cloud is the transformation enterprises are looking for whereas DevOps practices help achieve the transformation with speed and quality. When we talk about DevOps in the Hybrid Cloud environments, there are two main scenarios that can be adopted.
- Cloud hosted application delivery pipeline: When adopting DevOps one adopts a set of tools that create what is an Application Delivery pipeline. This includes tools such as an Integrated Development Environment (IDE), a Source Code Management (SCM) System, a Build tool, an asset repository, Test tools, an Automated Application Deployment tool, Environment provisioning tools and application monitoring tools, etc. All these tools can be hosted in the cloud and made available as a service. If you are using a PaaS such as IBM Bluemix as your application delivery platform, it includes these DevOps services.
- Cloud Hosted Environments: As applications are developed and delivered to production, they pass through multiple environments. These include development environments, multiple test environments, System Integration Test (SIT), User Acceptance Test (UAT), Performance Test, and potentially other pre-production environments (names and type vary from organization to organization), and of course Production. These environments may be spread over cloud and physical infrastructure, or even multiple cloud environments.
IBM has hosted offerings for the DevOps delivery pipeline that cover the first scenario, including its Collaborative Lifecycle Management as a Service (CLMaaS) and cloud hosted Rational Software Delivery Platform (SDP) tools available through IBMs business partner CloudOne. As mentioned before IBM Bluemix PaaS includes in it DevOps Services to deliver these capabilities in the PaaS.
For the purpose of this post, we will examine the second scenario in detail.
Application Delivery for Hybrid Cloud Environments:
Let’s examine a customer scenario I worked on recently. This customer, a large bank, was looking to host all their application delivery and production environments to the cloud. For cost management, they had decided to host their highly dynamic Development and Test (Dev-Test) environments on an off-premise’s ‘public’ or ‘virtual-private’ cloud, and host their Production environments on-premise on a private self-managed cloud. At the same time, their core business systems, in this case their core banking systems, all remained hosted on a Mainframe z/OS system. They are exposing their core Banking Applications to other applications that consume their services as REST APIs via an IBM DataPower gateway.
The goal of this organization is to be able to deploy complete business capabilities which comprises of services and middleware components deployed to the Production Cloud, and the back-end Mainframe. The primary challenge was to use a Deployment Automation tool that can deploy components and manage application configurations across all these multiple environments.
The second challenge was the need to deploy applications under development as they progressed through different environments in the delivery pipeline. These environments were hosted in a public cloud. As the production environments are hosted in a private cloud, there was a need to use ‘pattern’ technology that would allow for the provisioning of ‘production-like’ environments through the lower pre-production environments. These ‘patterns’ would allow for the production environments and the pre-production environments that match, albeit with different configurations and scale, to be managed and versioned as code. These patterns would evolve as the applications and the environments they were deployed on evolved through the development iterations.
Architecture for Application Delivery for Hybrid Cloud
Our discussion so far now leads us to designing architectures for application delivery for Hybrid Cloud which requires a deployment automation tool that supports multiple technologies comprising the hybrid cloud environment. As discussed before, multiple technologies ranging from off-premise, on-premise clouds to physical infrastructure may be included in these hybrid environments. IBM UrbanCode Deploy with Patterns provides such support.
IBM UrbanCode Deploy with Patterns supports two cloud pattern technologies:
- IBM Virtual System Patterns
- OpenStack HEAT/HOT templates
This allows IBM Urban
Code Deploy with Patterns to support both provisioning of cloud environments using these patterns, and the deployment of applications to these environments, whether to an on-premise or off-premise cloud. In addition, it supports application deployment to static environments on physical infrastructure – distributed and mainframe. The Cloud technologies supported by UrbanCode Deploy with Patterns include OpenStack Private Cloud, Softlayer, Amazon Web Services and VMWare vCenter. In addition, IBM UrbanCode Deploy with Patterns also supports deploying buildpacks to IBM Bluemix PaaS.It is apparent from the example above that hybrid cloud is changing the face of enterprises and organizations today. With the speed the market is changing today, one cannot depend on waiting for transformation. IBM can help enterprises with their Cloud and DevOps journey leveraging our unique cloud and DevOps expertise. On November 20, you can know more about Hybrid cloud strategies and interact with some of the industry experts like Rosalind Radcliffe (Distinguished Engineer, Chief Architect for CLM and DevOps, Member of the Academy of Technology at IBM), Richard Irving (Certified Client Technical Professional at IBM) and Wayne Monk (Vice President, Business Development and Alliances at Skytap) at the IBM DevOps Symposium in the New York area (Jersey City). We are streaming the first hour as a series of lightning talks given by few distinguished industry leaders – so join us live or on the web.
-
What is DevOps and Why Should CIOs Care?
Like the buzz-phrase: “cloud” or “big data”, DevOps is the new flavour of the month within the IT industry. You’ve probably heard of it by now – but many CIOs and heads of IT have not got to grips with what it is and are therefore reluctant to let developers take ownership and run their own operations.
A recent survey found that 23 per cent of respondents were not familiar with the term DevOps, and another 17 per cent thought it was all hype. DevOps is a method for improving the performance of the entire IT group. It is aimed at creating a better set of joined up IT outcomes for the business. As a methodology, DevOps allows all IT functions and activities, from cradle to grave, to be integrated into a seamless whole. It is an IT engineering philosophy and discipline that sees each part of the IT activity and value chain as being important to the delivery of IT products and services.
So why is it important? Well, DevOps is about culture, organisation, people, processes and tools. It helps directly address the historic silos that have existed between ‘development’ and ‘operations’. It also deals with the unhelpful and counter-productive behaviours that exist within different parts of the IT organisation that interfere with the smooth running of IT and impedes the delivery of high quality and cost-efficient products and services to the business.
DevOps provides a model for a more efficient and effective IT group; one that delivers more business value for less IT cost. By changing the way that different parts of the overall IT group operate together as a unified whole, DevOps improves the eventual outcomes for the business by making the ‘business of IT’ more efficient and more effective: “Better IT means better business”. Well-implemented DevOps initiatives will deliver rewards in terms of IT efficiency and quality improvements.
Whilst it can be a difficult sell to the business, the commercial benefits of a DevOps methodology applied to the ‘business of IT’ can bring significant improvements in productivity, consistency, flexibility and stability and must be championed by the CIO in order to get business buy in.
As promising as the DevOps movement is, especially for global organisations, some clear lessons have already emerged. Understanding these in advance could be the difference between success and failure for your DevOps initiative, and ultimately, your entire IT strategy. The alignment and collaboration of development and operations is key to a successful DevOps programme. Both groups must strive to achieve business agility through enabling change whilst preserving the stability of the existing IT infrastructure.
To achieve this the CIO must foster an environment where there is no longer two groups with opposing interests but one team that is pulling in the same direction and a culture where the individuals are cogniscent of each other’s responsibilities.
Get this right, and it will undoubtedly lead to better business.
About the Author
Paul Greer is Chief Software Architect at RedPixie. Paul works alongside the product and services teams to ensure products are high quality and enterprise ready. Paul also maintains a close relationship with RedPixie’s client base to ensure software services are delivered to a professional standard and exceed expectations. Prior to co-founding RedPixie, Paul spent 14 years in the financial services sector, holding technical leadership roles in business application development, innovation and architecture. -
USENIX LISA 2014 – DevOps grows in the enterprise
I just returned to frigid Boulder, CO from the USENIX LISA 2014 conference in sunny (not warm, but completely tolerable) Seattle, WA. For those unaware, LISA is the Large Infrastructure System Administration conference, which held its inaugural conference in 1986. A lot has changed in the past 28 years, but LISA continues to demonstrate (and serve) the need for members of large organizations to meet to share and learn the latest techniques, methods, and technologies. One might even venture to say that LISA is a conference that could be unaffected by new technologies and methodologies: until systems and networks can completely manage themselves, it seems highly likely that we won’t arrive at the grand unification methodology for systems administration, nor will we run out of new technologies.Interest in DevOps on the show floor was somewhat limited based on my many conversations. Some were true converts preaching to create DevOps converts and bolster the faithful, others were debating what DevOps means, and still others were unsure what it was. One even questioned the name, “DevOps,” and instead argued for the term “Operations Automation,” to avoid the unpleasantness involved in trying to figure out who “owns” DevOps: is it developers or systems admins? In my opinion (and I think the prevailing opinion) is that the whole business must own it, similar to the Lean Manufacturing trends of the ’90s. However, out of the many folks I spoke to, DevOps was a primary interest to only about 2-3% of the population.
That’s pretty representative based on my understanding of the general technical population, but that’s perhaps twice what I’ve heard quoted recently, around 1-1.5% overall technical penetration. I take that as a good sign: it says to me that DevOps is being adopted in larger organizations by folks who wield significant power within their organizations.
Out of the distinguished speakers at LISA this year, about 40% of them had some relation to DevOps, even if DevOps wasn’t specifically mentioned: the goal of automating previously manual processes to improve efficiency and time to market was a common theme. Believe me, that 40% number was not generous, others might argue that several other talks were also DevOps-related. LISA continues to show its relevance, bringing the latest methods and technologies into enterprise stalwarts. The future looks bright for large enterprises, with (in some cases) grizzled old system administrators, handily learning new tricks (even if they don’t want to admit it), thanks in part to the LISA conference.
-
Build security into your app development
Integrate security testing into your dev process now or else face cyber-attacks later
The way we develop software has been radically transformed in the last few years. Agility and speed are vital components for any company that wants to compete in the market. In order to achieve that it has proven necessary to break down barriers. The idea of separate silos with developers, operations, testers, and management working in isolation, sometimes even in opposition, is dated and flawed.
We’ve accepted the logic of bringing development and operations together with the DevOps movement. The recognition that testing is required earlier in the process has come. Now it’s time to apply the same logic to security. It’s time to bring InfoSec into the fold.
The security threat
Trends like cloud computing, big data, SaaS, and the BYOD end of the mobile revolution have created new opportunities for increased efficiency and productivity, but they also represent unique challenges for security. Sensitive data, financial information, and intellectual property are all exposed to risk when security is a secondary concern.
The issues that splashed back from the waterfall model for testing apply to security as well. Only passing applications through the QA process at the end of production, when time pressures to release were the greatest and it was too late to make significant changes, resulted in fixes being more expensive and software quality suffered.
If we don’t integrate security testing into the development process and make it part of the software development lifecycle now, then we run the risk of encountering exactly the same problem. We waste time retro-fitting functionality that should have been there in the first place, and we all know the pain of securing a hybrid system with legacy software that wasn’t designed with modern security threats in mind.
How to address security in software development
Automation for testing has enabled developers to move towards a continuous delivery system where new features can be rolled into live software as they are created. How do we ensure that security is maintained?
- Secure programming education is required to make sure developers are limiting and testing inputs, storing minimum data and encrypting, compartmentalizing the system, questioning requirements that may introduce security risks for little gain, limiting privileges, and analyzing and auditing the code.
- IAST (Interactive Application Security Testing) allows you to combine elements of static and dynamic techniques to run automated tests continuously on the software under development and see how it copes with malicious traffic. Since IAST monitors data inside the application it can pinpoint issues that might arise from real-world attacks, enable a useful assessment of the impact, and make it easier to remediate.
- Security analysts are needed to properly configure your tools and interpret the results. You can buy the best security tools in the world, but you have to know how to leverage them and act on the data. An external analysis can provide real insights that will boost application security.
- OWASP (The Open Web Application Security Project) is a great community where you can find innovative solutions to modern software security challenges. It can help you understand secure development standards and it’s packed with invaluable resources and advice from experts around the globe.
Bringing InfoSec on-board from the outset will help you build security considerations like this into your development pipeline. It will save time and money in the long term.
Ongoing research and modelling
It’s important to model potential threats and test for them, but you must be aware that new threats evolve and emerge all the time. Dedicated InfoSec employees will continually research and explore the new trends and risks in the security industry. Opening a direct line between DevOps and InfoSec enables them to pass along that wisdom and fold it into the mix when it’s relatively cheap and easy to do.
The security testing in your development pipeline is no more static than any other element. It has to be continually reviewed and modernized to ensure it continues to deliver results. Continuous real-time monitoring will deliver the oversight you need.
Building for the future
There’s no question that cyber-attacks will come, but if you prepare properly you can detect and nullify them with minimal effort. When considering the investment now, you must factor in the cost of lost confidence, post-mortem forensic investigation, and significant redevelopment to close any gaps in your defenses in the event of a future breach.
Once you have built solid foundations for security in your application development they will benefit every project going forward. Taking a long term view makes financial sense and results in better quality software.
About the Author
Michelle Drolet is founder of Towerwall, a data security services provider in Framingham, MA with clients such as Smith & Wesson, Middlesex Savings Bank, Brown University and SMBs. You may reach her at michelled@towerwall.com. -
Expanding DevOps Beyond Dev and Ops
I’ve been involved in software delivery for longer than I care to admit, and since DevOps has come onto the scene I’ve witnessed the deconstruction of silos that were once actively built to contain separate projects and team functions. Central to this deconstruction is collaboration between Dev and Ops, ushering in new innovation paradigms. What’s exciting is that in just the past five years since DevOps has started permeating organizations, other areas within the enterprise—including technical and marketing teams—have started to recognize the high-value that comes from embracing collaborative tools and processes. DevOps has quickly evolved from sideline projects to a state of mind rapidly adopted by leading companies across various business units.
DevOps and Culture
Accompanying this notion of greater collaboration and cross-silo participation is the need for new tools to help facilitate the pace of change and collaboration. Although there are no silver bullets, you can take steps and implement technologies to help make the odds of success more favorable.
Before DevOps, hostility between Dev and Ops teams was not unusual. Dev and Ops had conflicting goals: Development to introduce change and Operations to maintain stability. The common bond was a love of technology rather than delivering quality products to customers. In my experience, technology employees care more about the technology they can experiment with than the value something they’re working on brings to customers. However, in the five years that DevOps has been on the scene, those of us who follow DevOps closely have noticed staff on both sides becoming much more customer-focused.
Common Goals
Many organizations publish their core values and say that they expect employees to live up to those values. But are those values something that employees are measured against? In my experience, they are not. Organizational values tend to end up being words to make people feel good. But what if every manager in an organization had a common goal of delivering quality products or services to customers, with the goal defined in a measurable way? Then everyone in the organization contributes to delivering the products to customers. It’s a goal is not and should not be department specific.
The departments that often see considerable delays due to red tape are:
- Human Resources
- Budgeting
- Procurement
- Marketing
Any one of these departments can result in projects getting stuck in the mud, usually due to large amounts of lead time required to fulfill a request. Delivering quality products and services frequently obviously doesn’t mean having no process, but when things need to be done quickly, procedures should be flexible enough to accommodate such requirements. The need for agility applies to all parts of an organization. It’s not just a technology problem.
Consider trying to hire someone new to work on a project. Adding an employee could involve HR, Finance and Procurement, each with their own processes and requests for lead time. The multitude of steps in their respective processes frequently take weeks, if not months. This applies to almost any department of the organization and it’s highly likely that there are processes that add significant delays to reacting to business needs.
Setting common goals and measurements is a sure way to cut red tape. Employees don’t want to be the reason something could not be done. Processes will become streamlined and departments will collaborate out of necessity, if not for the good of the organization, then at least as an act of self-preservation.
While this high-level overview makes it sound simple, most organizations outside of Silicon Valley don’t set common goals for employees. It’s up to the Unicorns to spread the word.
Using Technology to Facilitate Collaboration
The DevOps movement resulted in advances in tools and technologies that aided in Dev and Ops collaboration, and took advantage of the opportunities created by this frictionless new environment.
In order for all departments in an organization to be able to take advantage of similar opportunities, other teams will need to look at ways to optimize their workflows and tool chains.
One area where tools have reduced friction and increased collaboration is code review. Code review has a lot in common with collaborating on any other kind of content. Code review used to be done with people sitting in a room with printouts of source code receiving feedback that often resulted in holy wars about coding styles. It wasn’t effective and that was even within the same department.
Tools now allow documents to be reviewed asynchronously with in-line comments, no matter where in the world the reviewers are located. It’s one of the few examples where communicating via text rather than voice lowers tensions, which is surprising given how easy it is to accidentally upset someone via email!
A great example of this is how my working style changed once I started working with Gene Kim, the award-winning author of The Phoenix Project: A Novel About IT, DevOps, and Helping Your Business Win and The Visible Ops Handbook. Rather than email documents back and forth, we’d collaborate via Google Docs. No matter what our schedules were or where we were in the world, we worked together effectively. The first time I worked like this with Gene changed the way I think about collaborating on non-technical content.
A key element of collaboration on a single set of artifacts is not having multiple copies in different systems in an organization. In fact, the 2014 DevOps Survey by PuppetLabs found that versioning production artifacts in one place was a key indicator of high-performing organizations. Imagine being able to store all artifacts involved in a project–Including requirements docs, marketing campaigns, and spreadsheets–In a single location accessible via tools that fit your needs. Collaboration, passing audits, applying for tax credits, and disaster recovery would be so much easier.
We’re not there yet. As mentioned, tools such as code review, Google Docs, wikis, version control systems, and email all help with collaboration but they are often still silos of information. Just as breaking down silos between teams has been revolutionary, breaking down the walls of data will one day have an equally profound impact.
About the Author
Before joining Perforce as a Technical Marketing Manager focused on Continuous Delivery and DevOps, Jonathan Thorpe held technical marketing roles at CFEngine, Serena, and Electric Cloud, where he specialized in DevOps culture and automation technologies. In his spare time you can find him reading, playing with all of the current video game consoles, dabbling with mobile software development, and playing with the Raspberry Pi. -
DevOps with purpose: ACT-ing with greater agility
This is Part Four in a four-part series. In Part One, I focused on the critical first step of defining DevOps with a purpose by thinking about DevOps in the context of your organization’s applications. In Part Two, I provided four tips to fostering a DevOps culture in your organization. In Part Three, I discussed the role of tools and how they can amplify (or constrain) individual and team abilities as well as work across teams.
In this final fourth part of the series, I’m going to weave the three previous topics of (A)pplications, (C)ulture, and (T)ools together and will show how you can ACT with purpose to start your own DevOps transformation. Your DevOps strategy should incorporate all three aspects of applications, culture, and tools. Given the breadth of those three areas, it’s easy to feel overwhelmed from the start. Here’s a tip I learned from a mentor a few years ago. Whether you are a CEO, VP, or team leader, smart leaders set a vision (often at least two or three years out) and then they make small decisions every week with the goal of generally moving the organization or team in the direction of the vision over time.
I recommend first that you set a vision for DevOps as it relates to your Applications, your Culture, and your Tools. Let’s begin with Applications. In the first article, I shared the concept of the pace-layering model developed by Gartner. To set a DevOps vision for your applications, place each of your current applications into one of the three layers – system of record, system of differentiation, or system of innovation. Then, do the same thing for any applications you are planning to add to your portfolio.
With that context, specific to your organization, formulate a strategic vision for how DevOps will apply to your systems in each layer. Will you seek to have DevOps principles for continuous integration and continuous delivery in place for all of your systems of innovation, and if so, by when? How about for systems of differentiation? Will DevOps be the norm for each of those applications or just some, and by when? Then ask yourself the same questions about your systems of record.
Next, turn to your culture. In Part two, I discussed the importance of baselining the degree of collaboration you have between the business, dev, and ops as well as how you view work (in workstations or in end-to-end flow terms). Then I challenged you to incentivize people to modify their behavior both with respect to their roles and as individuals. With this as context, define your DevOps cultural vision. What does collaboration between the business and development look like in the future? What does collaboration look like between development and operations? How about between dev, QA, and ops? And let’s not forget security.
How will you recognize when your culture is optimized for flow? For example, I recommend documenting a set of indicators that you can measure at regular intervals (e.g. quarterly). Here are three good ones. How much work-in-process (WIP) do we have for Application X today vs. how much was there a quarter ago? What is the average time it takes from the definition of a requirement to its deployment in production today vs. a quarter ago? How often did we make updates to Application X in production this quarter vs. last quarter?
Now, let’s consider tools. Take a baseline of the tools you are using today for each step of the typical DevOps tool chain from requirements management to source code control and bug tracking through to build management, automated testing, configuration management and deployment. And don’t forget project management tools too. What is the state of your tool investment in each of these areas? Do you have gaps? Do you have multiple tools in the same area, and, if so, is that okay or do you want to standardize on one? Do your tools vary based on the applications they are used with (see pace-layering discussion above)?
Then take an inventory of your collaboration tools across dev and ops. Are you still mostly using email and sharepoint sites? What role does instant messaging play in your dev and ops teams today? What about conference calls and meetings? Once you’ve baselined where you are today, set a vision for where you want to be in the future. Are you looking to reduce the need for conference calls and meetings, and if so by how much (time or average number of participants)? What issues are you running into around email and IM communications that need to be addressed and also for SharePoint? Have you looked at the emerging class of DevOps collaboration tools and concepts such as ITinvolve and ChatOps? What role can they play in streamlining communications, keeping people informed, and presenting information to staff so they don’t have to go hunting around for it in dozens of systems or call more meetings?
With this as background, define your DevOps tools vision by setting priorities for filling identified gaps and rationalizing existing investments, then set a general target time frame for each area.
Now that you’ve set a vision in each area – (A)pplications, (C)ulture, and (T)ools, it’s time to ACT. I recommend you first pick one or two applications to begin your DevOps journey. Ideally, they should be applications that are visible to the business and where you can lay claim to the impact DevOps can have on business goals like time-to-market and responsiveness to customer needs.
Once those applications are identified, take an inventory of all the people across your organization (including the business), those responsible for the development and delivery of the applications. Develop a specific plan for how you will educate those individuals on DevOps principles (e.g. attend DevOpsDays, read The Phoenix Project, read a sets of articles and watch a set of seminar or demo recordings), and also develop a specific plan for how you will incentivize behavior changes for their roles as well as what is expected of first line managers to take individual personalities, skills, and experiences into account.
Let me also stress that you shouldn’t ignore the people who aren’t participating in this first DevOps effort as part of your
communications. Educate them as well, for example through an all-hands meeting and an email from leadership. Explain why the company is investing in DevOps transformation, why these applications have been chosen, and your long-term vision across Applications, Culture, and Tools to realize the transformation. Help them understand where the organization is going and why, and how it may impact them in the future. Avoid the perception that “the best people were picked for this and you are not among them” or that this group of people is being given license to do things that others can’t (e.g. make changes outside of the standard change management process) – otherwise, you risk reducing morale, good people leaving, and even sabotage of your DevOps efforts.Once you’ve completed the above, map the relevant tools you have in place today for managing the development and delivery of these applications as well as the tools on your vision roadmap defined above that will be added or rationalized in the next six months.
By following the above recommendations, you will now have an action plan for the near term and a long-term vision you can use to guide daily decisions to move you closer to that vision. Lastly, don’t be rigid about your short-term plan or long-term vision, recognize that everyone is learning, including you, and that one of the key principles behind DevOps is the feedback loop!
Thank you for reading this series. I hope it’s been useful to you on your DevOps journey. If you have a question or just want to share a comment, you can do so here on devopsy.kinsta.cloud or send me an email at matthew@itinvolve.com.
Best wishes on your DevOps journey!
