devopsy.kinsta.cloud

Author: contributor

How Much QA is Enough in Software Development?
People working in quality assurance as testers, managers or engineers know what it’s like having to justify their existence; it’s been part of the job for many years. QA experts have had to fight the perception that what they do is a bottleneck, overkill and it slows everyone down.

Finally, however, QA is getting the attention that it deserves. Companies are gradually investing more in QA skills and tools. This began with the onset of Agile and DevOps—methodologies which have now infiltrated mainstream software development practices everywhere. Since those practices focus on speed, efficiency and quality, engineering leaders have grown to understand that great products are not achievable without adequate QA checks and balances. Another risk is the unflinching nature of high user expectations: Thanks to social media, one user complaint can quickly escalate into others across the web, creating an uncomfortable perception problem for the software provider.

Robust testing practices can prevent egregious flaws in the software that result in poor user experience and performance—or, worse, a breach. True, testing can sometimes get in the way of speed. Yet no matter how fast you release a product, if it doesn’t work well, no one’s going to buy it, much less use it.

Nevertheless, we can continually improve the QA practice to fit into modern development teams’ workflow. Testers need to work with developers and master automated testing, while developers need to sometimes engage in testing; everyone should think about the appropriate level and amount of testing. You can have too much of a good thing.

The Right QA Balance

The below ideas can help your organization reduce conflict between QA aficionados and those who would rather code and release as fast as possible.

1: Assess, analyze and report. Developers and testers alike are highly technical; they like numbers. Therefore, the more that the QA team can quantify the risk of an application, provide evidence as to the effectiveness of current QA practices and use metrics to show the need for more QA resources when warranted, the faster that everyone will get on board.

There are plenty of tools that can deliver common quality metrics such as total number of defects, defect leakage, test coverage, test pass/fail rates, test completion rates, code coverage and application uptime. It’s important to share and discuss these metrics throughout the development team and with product and business managers. What constitutes high quality for the development team may not be the case with executives who are closer to the end user/customer.

2: The gut check. After metrics, take into consideration the comfort level of your team and company concerning product flaws. In regulated spaces, or for applications that collect sensitive private data, the tolerance for bad code and security gaps is usually low. Gather feedback from different perspectives and roles in the organization for a balanced perspective on the QA practice and areas for improvement.

3: Get people involved. Quality assurance is no longer a separate activity from development; especially when you are practicing Agile and DevOps, the responsibility for quality is shared. Therefore, developers, business analysts and operations managers all should be doing their work with an eye to quality, delivering feedback regularly to the QA team. Provide opportunities and incentives for any team member to attend events and training related to software quality.

4: Streamline testing. Test automation tools are helping product teams be more comprehensive in testing, by automating low-risk test cases so that testers can focus their exploratory and manual methods on high-risk or new features. This is a positive development, although testers must take time to learn how to use the tools. Some other ways that can help teams be more efficient in testing include:
- Conducting more testing early in the sprint. This avoids the 11^th hour disaster of finding a major flaw right before the release deadline.
- Per Agile methods, dividing development tasks into smaller chunks with team members from different disciplines to ensure testing is being done all the time and with visibility by all.
- Using test driven development (TDD) or behavioral driven development (BDD), which requires that QA, developers and business stakeholders meet at the beginning of a project to agree upon requirements and test cases. Developers then write code against the test, until it passes. New tools are helping bridge the gap by writing requirements and user stories in plain English.
- Outsourcing part of the testing load during a major project, which can help increase coverage and reduce stress on your team. However, understand that too much outsourcing will undermine any efforts to develop QA as an internal best practice and “mindset” for software teams.
If your team is committed to QA as a best practice, not an afterthought, those at the top will need to invest properly in the right tools, training and people to get results. Misconceptions about QA still exist; contrary to the view of some DevOps experts, we can’t (and shouldn’t) automate every test. As well, even though developers are now testing the code they write, you still need experienced testers on your team. With the pace of innovation and the high stakes in the marketplace, organizations need more—not fewer—people engaged in testing.

For testers who love their job, as many of them do, it’s an exciting time to be in the field. Companies increasingly are seeing the value of QA and testers continually are gaining more tools and tactics to excel at helping their organizations produce really great software.

About the Author / Kyle McMeekin

Kyle McMeekin is Senior Sales Engineer at QASymphony. Prior to QASymphony, he was a programmer analyst at Cognizant Technology Solutions. Connect with him on LinkedIn.
May 8, 2017
From Architect to Operator: The Rise of Developers
Over the past five years, DevOps has transformed from a model for agile software development practices to the aspirational standard for functions across the IT spectrum. Previously, developers and IT Ops teams were distinct and siloed—developers built the application and IT Ops managed production environments—but now many in the industry have shifted to create a net-new liaison between the two, allowing applications to exist simultaneously in the worlds of development and production.

While everyone is focusing on the convergence and collaboration of the developer and the IT Ops person, however, the storyline is changing again. The focus is now shifting back to the developer (and his/her specific skill sets), as there’s an increased demand for a single team to manage the entire application life cycle. Now more than ever, the people who actually develop the systems are poised to take ownership of traditional IT operations functions, such as maintenance, performance and availability.

As IT continues to evolve, three trends point to the expanding and evolving role of the developer.
1. The decoupling of infrastructure from services. With the rise of public and private clouds, more of the technology stack is becoming abstracted away from infrastructure, and often offloaded to a provider. This has created challenges for how developers manage and deliver applications in this new environment. In the past, IT operations was responsible for ensuring things such as availability and performance, but now, developers can build those capabilities into the applications themselves.
2. The elimination of infrastructure setup. IT Ops roles have traditionally been the gatekeepers, and new technologies didn’t make their way into the data center if it didn’t pass their standards. Getting into the data center was just the first step, though, as lengthy deployment of physical infrastructure and provisioning of virtualized instances for applications would require substantial expertise and time. Now, developers can spin up services instantaneously, and infrastructure setup has been reduced to a simple configuration file. It is the developer who understands which resources the application will need, and with infrastructure reduced to a couple lines of code, the speed at which it can be applied is impossible to ignore.
3. Microservices create fault tolerance. How we’re building applications also greatly impact how they will run in the future. Historically, you would build a single monolithic service that would respond to every request from the users of that application. This approach meant that when a part of the service failed, everything would go down. However, when you abstract away the infrastructure, you gain the ability to decentralize the application or service into many loosely coupled microservices. This makes the application much more fault-tolerant and reduces the impact of an outage or shortage of resources. As such, developers can help ensure system continuity and performance by building resilience and monitoring capabilities into the software.
End-to-End Visibility and the Need for IT Ops Persists

While developers are certainly on the rise, the role of IT Ops remains uniquely necessary in today’s IT environment. IT Ops will always have the best end-to-end IT system perspective, encompassing the application environment as well as shared services and infrastructure (think backup, endpoint protection, single sign-on, DNS, etc).

Developers build visibility from the inside out. But because it’s happening in the app itself, there’s no outside-in look, and thus pieces of the stack go unmonitored. As a result, you can’t understand the holistic end-user experience—which, when it comes to application health, is both the start of the conversation and the end of the argument. IT Ops brings incredible value, offering end-to-end visibility with better monitoring and a focus on ensuring every transaction—both internal and external—goes smoothly. While it has become common for developers to instrument their applications, they don’t have nearly as much expertise in end-to-end service management and measuring performance across each component. Additionally, IT Ops workers tend to possess a great level of operational rigor; they’re better versed in dealing with third-party vendors; and they understand the entire system (the weakest link, what works, what fails, etc.), including back-end functions, in a way that developers simply don’t.

What Comes Next for Developers?

We’re already seeing how developers are taking greater control over traditional IT operations functions, but what comes next? Looking forward a few years, I believe the next frontier to monitor is the role developers will play in SecOps. Security attacks are about finding the path of least resistance, and as SecOps matures into active threat-hunting and breach detection, attackers will evolve their methods. If SecOps can solve current attack vectors—phishing, especially—tricking software into following unexpected code paths will once again be a reasonable route to a breach.

But there are ways to mitigate that during the development process, by building security into the code itself. As SecOps teams increasingly treat the environment as an ecosystem, they’ll see the application itself as a vulnerable piece of the attack surface; they may begin requiring secure development practices with a level of governance over development teams. As security increasingly becomes everyone’s job, everyone in IT must shift their mindset, and while development teams may be late to the party, SecOps will eventually extend a mandatory invitation.

About the Author / Eric Thomas

Eric Thomas serves as Director of Solutions Architecture for IT analytics company ExtraHop. Eric leads the Professional Services team, and draws on over 20 years of experience in IT operations. Before joining ExtraHop, Eric performed a variety of operational roles, most recently as Director of Advanced Engineering for Thomson Reuters, where he led a team of performance and availability specialists, supporting over 200 applications representing $2B in annual revenue. His prior experience includes Enterprise IT management, SaaS production operations and next-generation technology advocacy.
May 4, 2017
The Challenges of Digital Experience Monitoring
One of the great ironies of DevOps is that the better you are, the less likely it becomes that anyone will ever see your best work. If the code is supremely elegant, the app runs flawlessly, the servers do not lag and users and managers rarely say anything because that is exactly the experience they expect.

People become aware of you only when something goes wrong—and at that point, they are rarely full of compliments. The experience means everything to them.

Digital experience is the foundation of how online companies compete in the contemporary global market. That is why digital experience monitoring (DEM) has been recognized by Gartner as the most significant foundational process within application performance monitoring (APM). DEM is designed to improve not only your app but the entire experience that the user has in interacting with your brand online.

If you’re unfamiliar with the DEM category, check out my primer here.

DEM is the technological response to one of the oldest problems in business: not being able to be everywhere at once. On the digital side, the challenges of delivering the best possible performance across the widest scope fall into three spheres:
- The problem of divergent user tech
- The problem of network propagation
- The problem of splintered responsibilities
Let’s take a closer look at how DEM engages with each problem on its own turf.

Divergent User Tech

The mobile OS wars have settled on two champions for the time being: iOS and Android. As of February, those two account for 99.6 percent of mobile sales. Of course, from a DevOps perspective, nothing is that easy.

Within iOS, around 79 percent of devices are on iOS 10, and another 16 percent are on iOS 9. The other 5 percent are scattered across various earlier versions. On the Android side, version 6 Marshmallow has the largest share at 31.3 percent of the market, and there are a plethora of other versions out there. Then you have to worry about the mind-boggling variety of hardware.

For every combination of device and OS running your app, there are also users going to your mobile website on a handful of browsers. The numbers for February 2017 show 55.2 percent of the market is on Chrome, and 29.84 percent are looking at you through Safari. That leaves 14.96 percent of the market to be divided up among the default Android browser, Opera Mini, IE (at 1.02 percent, as hard as that is to believe) and many other contenders. It is significant to note here that the market leader did not even exist five years ago, so any apps you build now must be designed to work on browsers that haven’t been invented yet.

The second subset of problems within the user tech consideration is geographic variation. How your app runs on 4G LTE in the city will be very different from 3G in rural areas. While Atlanta was judged to be the fastest city in the nation with downloads averaging 16.9 Mbps, users in the capital of Washington, D.C., can only work at 9.2 Mbps.

The final subset covers issues related to the connection medium: Wi-Fi, cellular data, NFC, Bluetooth, ZigBee, WiMAX, HSPA, EV-DO, etc. It should be clear that there may be wide variations in digital experience based on the connectivity medium of various features within your app.

Network Propagation

It is not always the user’s fault. Old-school IT support tends to start with that assumption, but there are times when the problem originates within your own network or results from network latency generated by external errors.

Everything that is happening on your servers, from DDoS attacks to viral videos to presentations in the conference room, could impact the performance of your app. As microservices becomes the dominant design model, it has become more common to see distributed applications get hung up in communications across the network. How will you track down which team is at fault or even assign fault when communications between microservices break down? The bigger question is, how will you correlate what happened on the front end with what happened on the back end?

Splintered Responsibilities

The second set of problems implies the third, which has to do with establishing a more unified view of your digital experience beyond just the IT world or whichever departments are involved. If your app involves contractors or third parties, the bigger view is larger than your company itself.

It takes a paradigm shift to see your app as a unified tool. You have to assign a central command that takes a unified approach to keeping internal and external teams in agreement. This is the level where a team will determine the proper processes to launch at a moment’s notice and mitigate the damage from issues in real time.

A central command outpost running DEM is needed to ensure a swift diagnosis of the performance anomalies. Then the DevOps team can resolve the issues and deploy fixes with minimal disruption to whatever the end user wants to do. Ideally, you want to do it with zero downtime or at least an SLA of nine 9’s (0.0864 milliseconds of downtime per day).

About the Author / Kevin Goldberg

Kevin Goldberg works in marketing at SolarWinds working on the Monitoring Cloud team. Prior to SolarWinds, he worked at AppDynamics and WalkMe. Follow his ramblings on Twitter at @Kevin_Goldberg.
May 4, 2017
SauceCon Speaker Profile: Angie Jones, LexisNexis

Automated tests are an important part of continuous integration, but if they’re not reliable, they can cause more problems than they solve.

This problem is not new to Angie Jones, who works as a consultant automation engineer advising scrum teams on automation strategies and developing automation frameworks for countless software products.

Jones currently works for LexisNexis, an information provider for legal and regulatory businesses. LexisNexis recently began embracing agile and test automation companywide, and she was brought aboard to help the company master the change. She is consulting with multiple teams to help define their automation strategy, building automation frameworks and training their testers on automation techniques.

The biggest challenge so far has been finding different implementation strategies for different teams. “Automation strategies are not one-size-fits-all and should be customized for a given context,” she says.

Jones will bring her expertise to the upcoming SauceCon event and plans to present a talk titled, “The Build That Cried Broken: Building Trust in Your Continuous Integration Tests.” Her presentation will focus on automated tests that are part of a continuous integration process. These tests aren’t run privately on someone’s local machine, and the person running them simply gives a final verdict to the team and spares everyone the glitches and hiccups. In CI, the unreliability of these tests are exposed for all to see, and they are detrimental to the process.

In her talk, Jones will share how her team managed to turn their CI builds from an untrustworthy mess to a credible source of information. Since this isn’t a quick fix with a lot of cultural and mindset changes, she will discuss that as well.

She is very excited to give this talk at SauceCon because “people attending this conference are the ones who are invested in ensuring that they get CI right, and I want them to know that automated tests don’t have to be their downfall. Besides that, it’s a really fun talk that I think everyone will enjoy,” she says.

As a Master Inventor, Jones is known for her innovative and out-of-the-box thinking style which has resulted in 22 patented inventions in the United States and China. Angie shares her wealth of knowledge by blogging at angiejones.tech, speaking and teaching internationally at software conferences, serving as an adjunct college professor of Computer Programming and teaching tech workshops to young girls through TechGirlz and Black Girls Code. When she is not inventing something or helping people with test automation, she is reading urban fiction or soul line dancing!

Join her and other presenters at SauceCon, the first-ever Sauce Labs user conference. SauceCon is a three-day event filled with training, workshops, best practices and visionary content from the leading minds in automated testing. For more on upcoming SauceCon presentations or to register, visit the SauceCon website.

About the Author / Elizabeth Hechtel

Elizabeth is a passionate tech writer at TestObject. As a Junior Marketing Manager she is focusing on delivering the best quality content on mobile testing. Personally, she loves learning new languages and reading up on recent advancement of modern technology.

May 4, 2017
SauceCon Speaker Profile: Adam Carmi, CTO of Applitools

Carmi to present a live coding session on testing responsive sites

Testing responsive sites and applications can be quite a challenge for development teams. We are fortunate to have Adam Carmi, CTO of Applitools, presenting a session at SauceCon on “Advanced Test Automation Techniques for Responsive Apps and Sites.”

During Adam’s presentation, the audience will participate in a live coding session to learn how to effectively design responsive page objects, implement generic tests that work for all the layouts of their apps, control the browser’s viewport size to accurately target layout transition points, incorporate layout-specific assertions in their tests, and visually validate the correctness of their app’s layout.

Adam is passionate about everything related to software development and testing and is excited to share his knowledge with (and also learn from) similarly passionate practitioners.

As CTO and co-founder at Applitools, a cloud service for automated visual testing, Adam spends most of his time developing image processing algorithms. However, he loves testing of all sorts, from designing the product to maximize testability to unit and API testing and, of course, automated visual testing.

Adam regularly speaks about software testing at conferences around the world and organizes the Israeli Selenium Meetup group. He loves to ski and tries to stand on his hands as much as he can! When not working, he enjoys working out, watching movies and playing video games with his two children.

SauceCon, the first-ever Sauce Labs user conference, is a three-day event filled with training, workshops, best practices and visionary content from the leading minds in automated testing. For more on upcoming SauceCon presentations or to register, visit the SauceCon website.

About the Author / Rebecca Cramer

Rebecca Cramer is a writer and communications strategist specializing in technology. With more than 20 years of experience across disciplines including marketing, internal and executive communications at companies such as Dell and Quest Software, Rebecca is passionate about creating and sharing good content of all kinds. In her spare time, she enjoys spending time at the beach and hiking in her native Southern California.

April 27, 2017
Meeting the Need for Speed in Cyber Threat Response
In the very early days of the internet, hackers most likely were “lone wolves.” They might be an unhappy customer, a disgruntled employee or a tech-savvy youth who just wanted to see if he could breach a target’s defenses.

Occasionally, hackers might aspire to more devious crimes including identity theft, blackmail, theft of trade secrets, exposure of personal information or monetary theft. These types of hackers were annoying, unnerving and potentially dangerous, but they seldom caused widespread damages or serious financial losses. Today, however, cybersecurity professionals face an entirely new group of hackers and an escalating number of attacks—and security strategies and tools have not been able to keep pace.

The Modern Hackers

The greatest source of risk comes from well-financed, sophisticated hackers who often are connected to a government that sanctions and supports their activities. These groups constantly refine their already impressive skills and they are patient as well as persistent. They know how to circumvent defenses that rely on signatures or pattern matching, and they are adept at launching attacks that may take months to achieve fruition.

For example, sophisticated hackers may launch an upstream attack aimed at companies that make the products that others use for security, including SSL certificates and other digital credentials. These credentials are then used to steal money, intellectual data or other information from the group’s real targets. Hackers can also attack in stages, such as first going after information that will give them access to an organization’s network, allowing them to conduct whatever mayhem they want on their own schedule. They may install an encrypted “zero day” attack, for instance, or simply wait until a desirable piece of intellectual property is completed.

Why Speedy Responses Are Essential

The longer that a breach goes undetected, the more damage the incident will cause. It is a bit like having an undetected roof leak; the longer the leak allows water to penetrate beneath the roof, the more damage the water will do to the structure.

Unfortunately, most organizations are not doing a very good job of detecting breaches quickly. In 2014, the Verizon Data Breach Investigations Report revealed that 43 percent of all web application attacks were not discovered for months, and 85 percent of the point-of-sale intrusions went undetected for weeks. Other breaches have gone without detection for years, such as the Excellus breach that lasted 18 months and the allegedly state-sponsored “Project Sauron” attack that was in operation for almost five years.

When attackers have months or years to access a victim’s network, they have ample opportunity and time to inflict a substantial amount of damage. They can even expand their attack to infiltrate the networks of the original target’s customers or vendors. Interestingly, a study conducted by the Ponemon Institute found that approximately 33 percent of all attacks were not detected for two years — and two-thirds of the attacks were discovered by a third party rather than the compromised organization.

However, a report issued in June 2016 by the Business Continuity Institute indicates that many organizations are making progress when it comes to responding to cyberattacks. BCI surveyed 369 organizations in 61 countries. Approximately 66 percent reported that they had suffered at least one attack during the previous 12 months, and 15 percent reported that they had suffered 10 or more attacks. Roughly 31 percent claimed that they responded to attacks within an hour, but 19 percent said that they took at least four hours to respond. Approximately 24 percent were hit by a denial-of-service attack, while 45 percent suffered a malware attack; both forms of attack rendered the organization’s network inoperable or contaminated.

Responding Quickly to Attacks

Although most cybersecurity professionals know that response time is critical, not all of them understand the best ways to ensure that the speed is there when needed.
- A speedy response starts with an effective incident response plan. The plan should be based on a thorough assessment of threats, risks and potential failure modes, and it should be updated frequently and available to all parties.
- “Practice makes perfect.” Response teams should have ample opportunities to practice their tasks through frequent “dry runs.” Instead of wasting time trying to determine what they should do when an attack does happen, they can react reflexively.
- Automation can greatly reduce the amount of time required to respond to an incident. For example, without automation, if a breach occurs, staff members might have to manually check 5,000 or more endpoints. However, an automation platform can collect, analyze and report on the activity in much less time.
- Increase visibility across the different domains and systems. A recent survey of IT remediation teams conducted by the SANS Institute revealed that almost half felt that the lack of visibility was their main impediment to an effective, speedy response. Using a security information and event management (SIEM) solution for log aggregation and correlation can increase visibility across organization.
When the health of your organization is at risk, taking even an hour to respond to an incident can be far too long. Taking a day, a week or a month to detect and respond to a threat can do irreparable damage. With steps in place, organizations can meet their need for speed in addressing cyber threats quickly, to best protect their networks.

About the Author / Rishi Bhargava

Rishi Bhargava is co-founder and VP of Marketing for Demisto, a cyber security startup with the mission to make security operations “faster, leaner and smarter.” Prior to founding Demisto, he was vice president and general manager of the Software Defined Datacenter Group at Intel Security, and before Intel, he was vice president of product management for Datacenter and Server security products at McAfee, now part of Intel Security. He has more than a dozen patents in the area of computer security. He holds a BS in Computer Science from Indian Institute of Technology, New Delhi, and a Masters in Computer Science from University of Southern California, Los Angeles.
April 25, 2017
DevOps: The Innovation Power Couple
If “Dev” and “Ops” are the power couple of tech, “DevOps” is the holy matrimony that takes innovation from trainwreck to fast and repeatable. At most businesses, however, Dev and Ops still sleep in separate rooms and agree only on their mutual contempt.

Dev (Development) and Ops (IT Operations) usually are divorced or ready to serve the papers because their incentives clash. In Ops’ opinion, Dev is an idiot—always out late drinking with friends, he comes up with wild ideas and wants to implement them immediately. Dev can’t make anything work on real servers and can’t write scalable, stable code.

To Dev, Ops is a total code-blocker—she makes Dev go through “change management boards” and slows down progress. She lets the competition get to market first. So every day, Dev and Ops have an angry shouting match that ends in tears.

I wish I were exaggerating. My coworker Brandon Cipes once walked into an interview for a product development position. The interviewer broke down crying because the relationship between Dev and Ops had gotten that nasty. They refused to meet with each other, let alone be in the same room.

When Dev and Ops have a dysfunctional relationship, they struggle to communicate, collaborate and get results. Their organization will fail at innovation because it lacks the cultural, technological and procedural base to do otherwise.

The missing base is DevOps. It’s a unified strategy for Development and IT Operations, and implementing it can have the following effects:
1. Communication Becomes Unavoidable
Normally, Dev and Ops play “telephone” because they’re physically separated and have too many points of communication. Emails, IMs, grudgingly accepted meetings and passing conversations ensure Dev and Ops never get the full message.

DevOps fixes communication by assigning Dev and Ops personnel to the same teams. They sit in one room and work in a single squad (a.k.a. a pod or team) of no more than eight to 10 people. If they turn their heads, they see each other’s beautiful faces. Until promotions or career change do them apart, they can’t avoid talking and working toward common goals.

In essence, DevOps puts “cross-functional” teams together and thereby eliminates hand-offs. Fewer points of contact, less telephone, better communications.
1. Collaboration Gets Automated
For many Dev and Ops teams, “collaboration” is another word for bureaucracy. It’s TPS report-ish stuff characterized by change review board meetings and laborious documents that tell Ops how to deploy new code. Dev and Ops might accidentally (or intentionally) withhold information from each other.

In DevOps, collaboration is automated. Instead of change review board meetings, you use version-controlled software. The hand-offs from Dev to QA and QA to Ops are done by machine. A script deploys the code for Ops so there’s no risk of misfiring.

“Collaboration” is no longer an ordeal or choice. Documenting and sharing the squad’s collective knowledge is wired into the process.
1. Fast and Stable Releases
Normally, a software deployment is like a sloth trying to cross US 101: slow, awkward and soon to be roadkill. That’s all thanks to bad communication and bad collaboration.

With DevOps, you catch problems before they go live. Test automation prevents any need for manual checks. One person corrects the problem in a matter of minutes whereas before, a team of five might spend eight hours tracking down and fixing one bug. Ops feels confident that tested code won’t cause havoc.

Thus, the upshot of DevOps is speed and stability. Before, Dev and Ops delivered maybe a few times a year and faced regular breakdowns. DevOps can deliver multiple times daily without a hitch.

Say ‘I Do’

DevOps is a cultural, procedural and technological transformation. As in a marriage, both partners are responsible for good outcomes. Linked as they are on one squad, Dev learns to care about stability and Ops begins to value innovation.

If “agile” is about making businesses nimble and quick, DevOps is about applying agile principles to technological development and operations. Ultimately, it’s about beating your competition to the next big thing.

Dev, easy on the drinking escapades. Ops, live a little. Tie the DevOps knot and make some innovation babies.

About the Author / Zubin Irani

Zubin Irani is co-founder and CEO of cPrime, a full-service consultancy that focuses on agile transformations and project delivery solutions for hundreds of software teams, including more than 50 of the Fortune 100. Connect with him on LinkedIn.
April 21, 2017
How Cognitive Bias and ‘HumanOps’ Affect Dev Teams

Does your team take into account how the human element can affect development practices? It’s often overlooked, but critical to consider. Jason Hand of VictorOps, devopsy.kinsta.cloud’s Top DevOps Evangelist for 2016, writes and speaks on this particular topic often. In fact, he will be delivering a keynote titled, “Cognitive Bias and Its Impact on Continuous Improvement,” at the upcoming SauceCon conference, to be held June 6-8 in San Francisco.

SauceCon, the first-ever Sauce Labs user conference, is a three-day event filled with training, workshops, best practices and visionary content from the leading minds in automated testing.

“Cognitive bias is an important topic that is often left out of the conversation. The ‘HumanOps’ side of DevOps is a fascinating area to explore and I’m excited to get the conversation going for those in attendance at SauceCon,” he said.

Jason aims to help attendees understand the most important cognitive bias to watch out for, as well as ways to identify it in ourselves and others. Particularly when managing service disruptions and time-sensitive problems, understanding where we make common mistakes will go a long way to improving our response to outages, as well as continuously improving over time.

Jason serves as a DevOps Champion and advisor at VictorOps, where he advises on the future functionality and direction of the on-call and incident management platform, counseling the team on modern DevOps principles and new methods of delivering reliable software at scale.

In addition, Jason advises the growing worldwide community of IT professionals including Colorado’s front range, where he helps to organize DevOpsDays Rockies and the monthly DevOps meetups in Denver. Co-host of “The Community Pulse,” a podcast on building community within tech, Jason is dedicated to the latest trends in technology, sharing the lessons learned and helping people continuously improve.

In 2016, Jason was named “DevOps Evangelist of the Year” by devopsy.kinsta.cloud. He is the author of two books on the subject of ChatOps and a forthcoming O’Reilly book on post-incident reviews, as well as a regular contributor of articles to Wired.com, TechBeacon.com and many other online publications.

For more on upcoming SauceCon presentations or to register, visit the SauceCon website.

About the Author / Rebecca Cramer

Rebecca Cramer is a writer and communications strategist specializing in technology. With more than 20 years of experience across disciplines including marketing, internal and executive communications at companies such as Dell and Quest Software, Rebecca is passionate about creating and sharing good content of all kinds. In her spare time, she enjoys spending time at the beach and hiking in her native Southern California.

April 20, 2017
9 Lessons Learned: Getting Value Stream Insights in Only 2 Days
Gene Kim, Jez Humble and other experts say that understanding and improving your value stream is really what it takes to bring your software delivery organization to the next level. While almost every IT organization has this goal, actually knowing how to make it happen can be paralyzing, because it seems like such a daunting undertaking when trying to connect all of your systems tools and processes.

Recently, we participated in an innovative approach that proved that you can take concrete and measurable steps and see real outcomes in only two days.

Nationwide constantly strives to innovate and bring real change to its IT organization. The insurance and financial services company has zeroed in on improving its value stream. To begin the initiative, Nationwide developed a two-day workshop where the right disciplines would come together and determine the best way to make this happen. But the real innovation in this idea was that the organizers did not just talk about what to do. Instead, the end goal was to come up with at least one key metric the team could show to the Nationwide CIO by the conclusion of the workshop. This approach made the effort real, concrete and focused. While it was difficult, it worked.

Here are nine important lessons learned:

Lesson 1: Start with one artifact—specifically, defects. Measuring cycle time is extremely difficult because there are so many moving parts. To have a tangible outcome, start with defects because they tend to be the most well-understood artifacts and have relatively defined workflows and processes that people know and understand.

Lesson 2: Commit to having a real metric based on real data. All too often organizations take the “boil the ocean” approach. When asked what metric they want to start with, the answer is typically, “Everything.” That is precisely the wrong approach. Aiming for everything is likely to get you nothing. Focus on a tiny amount to start. First understand, then improve, your software delivery value stream. Not only should you start small, but for IT value streams, start specifically with a certain artifact (see Lesson 1). And chose only one metric for that artifact, because less is truly more in this case.

Lesson 3: Without a multi-disciplinary approach, you will fail. This part is extremely difficult. To be successful you will need the following mix of skills:
- Domain Expert: This person knows this part of the value stream inside and out. She is a practitioner in that area of IT and truly understands the who, what, when, where and why of that particular area of the value stream.
- Process Analyst: This person knows how the various people, systems and resulting artifacts are intended to operate together for the complete process. Often this person is closely aligned with the domain expert, but typically is not one of the practitioners actually participating in the process.
- Data Warehouse Specialists: These are the people responsible for populating and managing the data marts or data warehouses. They need to have significant data transformation capabilities and intimate knowledge of the required data structures and formats the organization requires.
- Report Writers: As often happens with the expertise required to wrap up a process, this role is sometimes overlooked at the beginning. This is not the right approach. The person who will report on the project should be involved from the very start.
Some people might start by asking, “Don’t we need a data scientist?” After all, that is a title of authority when it comes to metrics. If you have a data scientist on staff, by all means, get them involved. But if you don’t, you can rely on someone with database skills—primarily, skills around writing advanced SQL (as old fashioned as that may seem). Most of this work will not require advanced data scientist skills.

But the real challenge is to get all of those disciplines in the same room at the same time. This is no easy feat. Leadership must provide the necessary resources to ensure that each one of the roles is represented. Because if any of these roles are left out, you will not succeed.

Lesson 4: Work in parallel. The process and domain experts can work on identifying the artifacts that must be used and manually identifying and mapping where the artifact originates, what systems touch the artifact when, and all the requisite status changes and their meanings. That work will inform the data warehouse team and report writers, but the data and report writers are actually faced with a very different problem: They must determine how to get the data, regardless of the identified artifact, and transform the raw data into a usable format that report writers and BI analysts can work with in their visualization tools. By having both teams work in parallel, with identified checkpoints along the way, you will be more efficient when it comes to the three required deliverables.

Lesson 5: There are always three required deliverables for each value stream cycle metric you are interested in.
- The artifact(s) of interest (defect, story, requirement, code commit, etc.)
- The following data:
  - All required status changes and dates of changes,
  - Data to relate artifacts to each other (for the joins)
  - Any ancillary information needed to be able to slice and dice the data for the final visualization.
- The final visualization showing the metric (which is typically not one metric but rather a macro metric that has drill down capabilities and other perspectives on the metric).
Lesson 6: Work both forward and backward. It is always easier to get started when you know where you want to end up. In this lesson, identifying the end goal is key. Once you have identified the “end” then you can go to the beginning and determine if the end goal is actually reachable. You also can identify any holes that exist between the beginning and your end goal. In this use case, the backward story looks like this:
- End Goal: A visualization (graph/chart) showing the metric of interest. The idea is to mock up exactly what you want the end result to look like. Create the mock up in the actual BI tool you intend to use as if you already have all of the necessary data at your fingertips.
- Step 4: Provide the data needed to create the visualization—it must be formatted in a way that will enable the report writers to easily and efficiently create the visualization that is your end goal. Hint: start by creating a spreadsheet that contains the exact data structure and formats desired. You will use fabricated data to start. The idea is to make sure the data is structured properly to reach the end goal (the visualization).
- Step 3: Create the interim views and tables that will enable you to get to Step 4
- Step 2: This step is where you collect the raw data from the actual systems involved
- Step 1: (The Beginning) Identify the artifact and all of the systems that artifact flows through.
Lesson 7: Invite outsiders to participate. It’s difficult to see the forest for the trees in your own organization. You need outsiders who are willing to ask the questions, probe in challenging areas, push the boundaries—people who have a different point of view because they are not in the game. This does not mean outsiders should be the workshop facilitators. They could be, but it is more important for them to actually participate in the sessions and help define and create the deliverables.

Lesson 8: Hold the workshop in a different space than you work in every day. Don’t use the conference room that you typically use. Bring the team to a completely different environment—one that inspires creative thought and innovation. It may seem like a fluffy requirement, but you will learn through workshop participant feedback that location has a major impact when it comes to productivity and focus.

Lesson 9: Get executive leadership buy-in. The executive VP was genuinely impressed with what the team accomplished in only two days. And the positive feedback and kudos the team received has probably fueled their innovation engines for months to come. To feel a sense of accomplishment and be recognized by the executives in your organization has significant morale impact.

Improving your value stream is no small feat. But by designing and executing on a workshop like this, you will see benefits and clear wins immediately. By the end of our two days, we had a prototype of defect cycle time and a defined repeatable process. Equally as important, the people involved were given the opportunity to fail fast and pivot where needed, which inspired innovative thinking and creativity. And, being able to present to the CIO real progress with real results in two days’ time fueled institutional buy in and momentum to drive a long-term value stream improvement program.

About the Author / Nicole Bryan

Nicole Bryan is Vice President of Product Management at Tasktop Technologies. Nicole has extensive experience in software and product development, focused primarily on bringing data visualization and human considerations to the forefront of Application Lifecycle Management. Most recently, she served as director of product management at Borland Software/Micro Focus, where she was responsible for creating a new Agile development management tool. Prior to Borland, she was a director at the New York Stock Exchange (NYSE) Regulatory Division, where she managed some of the first Agile project teams at the NYSE, and VP of engineering at OneHarbor (purchased by National City Investments). Nicole holds a Master of Science in Computer Science from DePaul University. She is passionate about improving how software is created and delivered – making the experience enjoyable, fun and yes, even delightful.
April 12, 2017
Challenges in DevOps Standardization
Many organizations still face challenges in their Devops adoption journey enterprise-wide, especially when there is no standardization in tools, non-repeatable processes and adhoc practices and culture barriers. Here are some of the challenges in standardization and possible solutions to achieve standardization in DevOps as part of DevOps transformation journey.

Business Case

A survey conducted within multiple customer teams revealed they face challenges in standardization of tools and streamlining the process gaps due to lack of end-to-end orchestration, manual intervention, legacy practices, inconsistent environments, longer provisioning time and several other factors. Taken together, they necessitate standardization in DevOps across the enterprise to make it repeatable, predictable and consistent.

Challenges in Standardization of DevOps

Tools Challenges
1. Multiple tools to evaluate with one or more overlapping features in the tools, learning curve involved to evaluate/do proof of concept and choosing the tools with short-term goals instead of long-term perspective.
2. Limited knowledge of tools, used from the perspective of automation in their specific area, resulting in siloed automation and using manual, ad hoc process.
3. Lack of out-of-box integration with most tools used in the CD pipeline and teams investing time to build custom integrations.
Process Challenges
1. There is no central DevOps team who can evaluate and recommend the right set of tools to be used, with individual project teams/groups try to set up their desired tools and serve their teams (Dev/QA/Ops), with no clarity on availability of infrastructure that can be shared across groups.
2. There is no defined process framework detailing the different tenets in DevOps, including what activities and approach are needed to move to desired state, leaving teams to achieve their desired state of automation without a structured approach.
3. There is a lack of knowledge on defining right KPI for each DevOps capability and how to measure the metrics to determine efficiency of the DevOps automation process and identify scope of optimization.
4. There is a lack of SME knowledge on the application, application stack and environment management flowing between Dev and Ops teams to arrive at a standardized process.
5. Multiple vendors are managing the different stages of an application and each following their own defined process for development life cycle.
Cultural Challenges
1. Standardization on metrics, process, practices, governance and organization-level control over DevOps implementation is challenging without a central authority (a central DevOps team) driving these practices across the enterprise.
2. Teams operate in silos and follow their own DevOps practices and think in silos without having a holistic view, resulting in broken processes that are inconsistent.
3. Teams still lack training in DevOps on the tools, practices to be followed in CI, continuous testing, continuous deployment and operations. Lack of DevOps skilled resources/expertise brings challenges in implementing end-to-end automation.
4. DevOps is considered a concept, with every DevOps implementation unique for every organization and no standard approach for adoption of DevOps, causing confusion where to start and how to start. In addition, there is a conflict of interest and power of influence from multiple stakeholders while standardizing on tools.
Proposed Solution(s)

DevOps Adoption Framework (DAF)

DevOps adoption framework (DAF) is a comprehensive framework for due-diligence, tools support and solution accelerators to create systematic and well planned-out transition road map and implement a continuous delivery toolchain and overcome the common challenges in DevOps implementation.

Choosing the Right Set of Tools and Stack

Tools selection should be based on the requirements and what best suits the need, decided based on various factors discussed in the attached whitepaper, and using one of the methodologies such as points given for out of box features and different factors/constraints.

DevOps Automation Platform with Template-Based Infrastructure and Toolchain Orchestration

DAAS (DevOps as a service) is an automation platform, which serves as a central platform to drive standardization across tool use, provisioning infrastructure, toolchain, managing the life cycle of VMs, bringing in central authority and governance, defining the KPI, monitoring and measuring the metrics, monitoring end-to-end pipeline giving real time analytics, providing self-service capabilities across multiple teams, bringing in audit and compliance, continuous improvement and drive consolidation and standardization.

Future Direction
- Increased focus on standardization of DevOps metrics with analytics and monitoring to identify the bottlenecks early and big data used for predictive analytics through the delivery pipeline.
- Rapid DevOps adoption across the enterprise, with standardization and consolidation critical factors for scale, compliance and utilization with self-service DevOps automation platform enabling end-to-end pipelines.
Conclusion

The critical success factors for enabling DevOps adoption across enterprise needs focus on central DevOps team driving DevOps adoption with governance enabled with DevOps adoption framework and consolidation of the processes, standardization on metrics and tools, enabled with a DevOps automation platform.

For a deeper look at the survey, download the white paper here.

About the Author / Lavanya Subbarayalu

Lavanya Subbarayalu is Senior Architect working with Technology Office in HCL Technologies. She has expertise in Azure, DevOps consulting & Microsoft technologies. She is associated with DevOps COE, working on design and Development of DevOps solutions and consulting tools. Connect with her on LinkedIn.
April 5, 2017