devopsy.kinsta.cloud

Tag: automation

How Much QA is Enough in Software Development?
People working in quality assurance as testers, managers or engineers know what it’s like having to justify their existence; it’s been part of the job for many years. QA experts have had to fight the perception that what they do is a bottleneck, overkill and it slows everyone down.

Finally, however, QA is getting the attention that it deserves. Companies are gradually investing more in QA skills and tools. This began with the onset of Agile and DevOps—methodologies which have now infiltrated mainstream software development practices everywhere. Since those practices focus on speed, efficiency and quality, engineering leaders have grown to understand that great products are not achievable without adequate QA checks and balances. Another risk is the unflinching nature of high user expectations: Thanks to social media, one user complaint can quickly escalate into others across the web, creating an uncomfortable perception problem for the software provider.

Robust testing practices can prevent egregious flaws in the software that result in poor user experience and performance—or, worse, a breach. True, testing can sometimes get in the way of speed. Yet no matter how fast you release a product, if it doesn’t work well, no one’s going to buy it, much less use it.

Nevertheless, we can continually improve the QA practice to fit into modern development teams’ workflow. Testers need to work with developers and master automated testing, while developers need to sometimes engage in testing; everyone should think about the appropriate level and amount of testing. You can have too much of a good thing.

The Right QA Balance

The below ideas can help your organization reduce conflict between QA aficionados and those who would rather code and release as fast as possible.

1: Assess, analyze and report. Developers and testers alike are highly technical; they like numbers. Therefore, the more that the QA team can quantify the risk of an application, provide evidence as to the effectiveness of current QA practices and use metrics to show the need for more QA resources when warranted, the faster that everyone will get on board.

There are plenty of tools that can deliver common quality metrics such as total number of defects, defect leakage, test coverage, test pass/fail rates, test completion rates, code coverage and application uptime. It’s important to share and discuss these metrics throughout the development team and with product and business managers. What constitutes high quality for the development team may not be the case with executives who are closer to the end user/customer.

2: The gut check. After metrics, take into consideration the comfort level of your team and company concerning product flaws. In regulated spaces, or for applications that collect sensitive private data, the tolerance for bad code and security gaps is usually low. Gather feedback from different perspectives and roles in the organization for a balanced perspective on the QA practice and areas for improvement.

3: Get people involved. Quality assurance is no longer a separate activity from development; especially when you are practicing Agile and DevOps, the responsibility for quality is shared. Therefore, developers, business analysts and operations managers all should be doing their work with an eye to quality, delivering feedback regularly to the QA team. Provide opportunities and incentives for any team member to attend events and training related to software quality.

4: Streamline testing. Test automation tools are helping product teams be more comprehensive in testing, by automating low-risk test cases so that testers can focus their exploratory and manual methods on high-risk or new features. This is a positive development, although testers must take time to learn how to use the tools. Some other ways that can help teams be more efficient in testing include:
- Conducting more testing early in the sprint. This avoids the 11^th hour disaster of finding a major flaw right before the release deadline.
- Per Agile methods, dividing development tasks into smaller chunks with team members from different disciplines to ensure testing is being done all the time and with visibility by all.
- Using test driven development (TDD) or behavioral driven development (BDD), which requires that QA, developers and business stakeholders meet at the beginning of a project to agree upon requirements and test cases. Developers then write code against the test, until it passes. New tools are helping bridge the gap by writing requirements and user stories in plain English.
- Outsourcing part of the testing load during a major project, which can help increase coverage and reduce stress on your team. However, understand that too much outsourcing will undermine any efforts to develop QA as an internal best practice and “mindset” for software teams.
If your team is committed to QA as a best practice, not an afterthought, those at the top will need to invest properly in the right tools, training and people to get results. Misconceptions about QA still exist; contrary to the view of some DevOps experts, we can’t (and shouldn’t) automate every test. As well, even though developers are now testing the code they write, you still need experienced testers on your team. With the pace of innovation and the high stakes in the marketplace, organizations need more—not fewer—people engaged in testing.

For testers who love their job, as many of them do, it’s an exciting time to be in the field. Companies increasingly are seeing the value of QA and testers continually are gaining more tools and tactics to excel at helping their organizations produce really great software.

About the Author / Kyle McMeekin

Kyle McMeekin is Senior Sales Engineer at QASymphony. Prior to QASymphony, he was a programmer analyst at Cognizant Technology Solutions. Connect with him on LinkedIn.
May 8, 2017
DevOps is a Lifeline to Connected Mainframe ROI
A new IDC white paper sponsored by IBM and CA Technologies reveals the adoption of DevOps is a hallmark of those organizations that are actively modernizing and integrating legacy systems such as a mainframe to support digital transformation.

Early adopters are blazing a path for faster innovation and greater ROI.

DevOps Delivers Mainframe Modernization and Integration

I’m encouraged to see DevOps recognized as an important element in realizing a “connected mainframe” vision. Respondents demonstrated a continued commitment to modernizing their mainframe environment.

The IDC study confirmed that leveraging Java on the mainframe was one of the most popular approaches. In fact, I’m constantly surprised at how few managers realize that these days, Java and Git can be used by developers in such a way that allows them to completely separate themselves from the mainframe “green screen” environment when coding or maintaining applications.

DevOps automation has also helped organizations overcome many of the cultural and operational challenges to a fully end-to-end enterprise by aligning application development tools and orchestration chains across multiple platforms and teams.

Adopt a Business-First Approach

Customers I’ve spoken with are experimenting with DevOps for mainframe tools and practices, with many piloting new approaches across the life cycle. They’re immediately becoming more agile, realizing shortened development cycles, improved collaboration between distributed and mainframe teams and faster MTTR that maximizes availability. In all, these early adopters have benefited from increased speed to market, improved business agility and better platform economics.

The IDC study concurs, identifying more effective delivery of applications and services as a powerful cost-saver, with savings of $58,900 per application achieved through increased developer productivity and day-to-day support efficiencies. By helping mainframe organizations to realize improvements at every stage of the application life cycle—from build through test, deploy and operate—DevOps delivers compelling benefits; for example, from our recent study Accelerating Velocity and Customer Value with Agile and DevOps:
- Advanced DevOps users reduce the time taken to develop, test and release new applications by 42 percent.
- Advanced agile users reduce their time to decide and act on new opportunities by 40 percent.
- Advanced agile users accelerate the time to develop, test and release new apps by 36 percent.¹
Excited? I encourage you to catch up on the full IDC White Paper here.

— Aline Gerew
May 3, 2017
Is Site Reliability Engineering the True ‘Ops’ in DevOps?

I recently had the privilege of attending the SRECon conference in San Francisco. It’s in its third year and 650 attendees gathered to learn and share their experiences, insights and site reliability engineering (SRE) practices.

Most SRECon attendees either identified as SREs or were there to learn more about this new role and approach. While there was strong representation from the “unicorn” community (Google, Shopify, Twitter, Netflix and Linkedin), enterprises such as Capital One and Comcast were among the many exhibitors actively seeking to recruit SRE talent. The conference spirit was infused with enthusiasm, a willingness to share, innovative thinking and grassroots advice. It reminded me very much of my first DevOps Days in 2012. There is no doubt that interest in SRE is rapidly growing and worthy of attention by enterprise IT.

The practice of site reliability engineering was born out of the book by the same name from Google. Interestingly, the SRE movement emerged separately from the DevOps movement—although there is little doubt that they are part of the same IT spectrum with similar customer value-driven goals. DevOps focuses on engineering continuous delivery to the point of deployment; SRE focuses on engineering continuous operations at the point of customer consumption. Both domains rely on sharing, culture, metrics and automation. Both require human and automated resources to ensure a seamless value stream and exceptional customer experience.

The role of the SRE is rapidly gaining momentum. Part systems administrator, part second tier support and part developer, SREs are encouraged to be inquisitive, to acquire new skills, to ask questions, to solve problems and to embrace automation. I asked several of the exhibitors what types of skills they were looking for in new SREs. The answers almost unilaterally included the ability to code as well as an unwavering thirst for knowledge and improvement—to find answers, solve problems and ensure the highest levels of reliability and availability.

I also can see a clear bridge between SRE with IT service management (ITSM). Many of the presenters spoke about the handling and management of late-night incidents, escalations, root cause analysis, service level objectives, availability metrics and other topics addressed in ITSM frameworks such as ITIL. The tools and process definitions may be different, the opportunity to instill agility into many of the ITSM processes to meet changing demands is certainly apparent.

What are the risks? To my mind the biggest risk is if SRE, DevOps, ITSM, Agile SDLC and other frameworks do not align and remain siloed outposts. SRE and DevOps are two sides of the same coin. Let’s hope that the SRE momentum catches up to DevOps and that enterprises begin to integrate both sets of practices as the norm.

— Jayne Groll

April 26, 2017
Meeting the Need for Speed in Cyber Threat Response
In the very early days of the internet, hackers most likely were “lone wolves.” They might be an unhappy customer, a disgruntled employee or a tech-savvy youth who just wanted to see if he could breach a target’s defenses.

Occasionally, hackers might aspire to more devious crimes including identity theft, blackmail, theft of trade secrets, exposure of personal information or monetary theft. These types of hackers were annoying, unnerving and potentially dangerous, but they seldom caused widespread damages or serious financial losses. Today, however, cybersecurity professionals face an entirely new group of hackers and an escalating number of attacks—and security strategies and tools have not been able to keep pace.

The Modern Hackers

The greatest source of risk comes from well-financed, sophisticated hackers who often are connected to a government that sanctions and supports their activities. These groups constantly refine their already impressive skills and they are patient as well as persistent. They know how to circumvent defenses that rely on signatures or pattern matching, and they are adept at launching attacks that may take months to achieve fruition.

For example, sophisticated hackers may launch an upstream attack aimed at companies that make the products that others use for security, including SSL certificates and other digital credentials. These credentials are then used to steal money, intellectual data or other information from the group’s real targets. Hackers can also attack in stages, such as first going after information that will give them access to an organization’s network, allowing them to conduct whatever mayhem they want on their own schedule. They may install an encrypted “zero day” attack, for instance, or simply wait until a desirable piece of intellectual property is completed.

Why Speedy Responses Are Essential

The longer that a breach goes undetected, the more damage the incident will cause. It is a bit like having an undetected roof leak; the longer the leak allows water to penetrate beneath the roof, the more damage the water will do to the structure.

Unfortunately, most organizations are not doing a very good job of detecting breaches quickly. In 2014, the Verizon Data Breach Investigations Report revealed that 43 percent of all web application attacks were not discovered for months, and 85 percent of the point-of-sale intrusions went undetected for weeks. Other breaches have gone without detection for years, such as the Excellus breach that lasted 18 months and the allegedly state-sponsored “Project Sauron” attack that was in operation for almost five years.

When attackers have months or years to access a victim’s network, they have ample opportunity and time to inflict a substantial amount of damage. They can even expand their attack to infiltrate the networks of the original target’s customers or vendors. Interestingly, a study conducted by the Ponemon Institute found that approximately 33 percent of all attacks were not detected for two years — and two-thirds of the attacks were discovered by a third party rather than the compromised organization.

However, a report issued in June 2016 by the Business Continuity Institute indicates that many organizations are making progress when it comes to responding to cyberattacks. BCI surveyed 369 organizations in 61 countries. Approximately 66 percent reported that they had suffered at least one attack during the previous 12 months, and 15 percent reported that they had suffered 10 or more attacks. Roughly 31 percent claimed that they responded to attacks within an hour, but 19 percent said that they took at least four hours to respond. Approximately 24 percent were hit by a denial-of-service attack, while 45 percent suffered a malware attack; both forms of attack rendered the organization’s network inoperable or contaminated.

Responding Quickly to Attacks

Although most cybersecurity professionals know that response time is critical, not all of them understand the best ways to ensure that the speed is there when needed.
- A speedy response starts with an effective incident response plan. The plan should be based on a thorough assessment of threats, risks and potential failure modes, and it should be updated frequently and available to all parties.
- “Practice makes perfect.” Response teams should have ample opportunities to practice their tasks through frequent “dry runs.” Instead of wasting time trying to determine what they should do when an attack does happen, they can react reflexively.
- Automation can greatly reduce the amount of time required to respond to an incident. For example, without automation, if a breach occurs, staff members might have to manually check 5,000 or more endpoints. However, an automation platform can collect, analyze and report on the activity in much less time.
- Increase visibility across the different domains and systems. A recent survey of IT remediation teams conducted by the SANS Institute revealed that almost half felt that the lack of visibility was their main impediment to an effective, speedy response. Using a security information and event management (SIEM) solution for log aggregation and correlation can increase visibility across organization.
When the health of your organization is at risk, taking even an hour to respond to an incident can be far too long. Taking a day, a week or a month to detect and respond to a threat can do irreparable damage. With steps in place, organizations can meet their need for speed in addressing cyber threats quickly, to best protect their networks.

About the Author / Rishi Bhargava

Rishi Bhargava is co-founder and VP of Marketing for Demisto, a cyber security startup with the mission to make security operations “faster, leaner and smarter.” Prior to founding Demisto, he was vice president and general manager of the Software Defined Datacenter Group at Intel Security, and before Intel, he was vice president of product management for Datacenter and Server security products at McAfee, now part of Intel Security. He has more than a dozen patents in the area of computer security. He holds a BS in Computer Science from Indian Institute of Technology, New Delhi, and a Masters in Computer Science from University of Southern California, Los Angeles.
April 25, 2017
CD 101 Podcast: The What, How and Future of Continuous Delivery
Now that we’ve arrived at Episode 4 of our CD 101 Podcast, let’s address some basic concepts—like the what and how of it all and the future.

Wait … perhaps we could—or should—have touched on these key topics earlier. Or not. At least not the future part? Anyway …

While this (mostly) backwards-looking orientation may seem a bit odd, I’d argue that we’re actually navigating the conversation on precisely the correct path. Let’s rewind for a moment to look at where we’ve come from.

Episode 1: We highlighted many of the fundamentals of this thing called CD, yet we did so from more of a conceptual and best-practices standpoint. While the ideas and processes involved do resonate in the real world, the topics we cover in this episode are bit closer to the water-cooler conversations we’re hearing about CD as it grows in practical adoption.

In Episode 2: We covered the whole how do you “realize agility” piece of the conversation. This is important and a precursor to some of these other basic questions. The entire process of digital transformation begins with getting agile and you can’t truly adopt continuous delivery without doing so; therefore, this was another good place to feed today’s discussion.

In Episode 3: We looked at the prospect of re-inventing the “Modern Software Factory” with agile development and continuous delivery. OK, so, perhaps we were getting a bit ahead of the basics here, but I’d maintain that this is an important piece to consider upfront that builds on these other building blocks. As with the topics covered in Episode 2, it is critical to understand the big picture challenges and benefits before you begin to get your hands dirty.

And now for some nitty gritty.

In Episode 4, we hear from Shamim Ahmed, CTO of Continuous Delivery Solutions at CA Technologies, as he answers the nine burning questions he most often hears in speaking to enterprise practitioners as they look to employ DevOps and Continuous Delivery practices in the real world.

So, while these previous conversations were material, this is a crucial set of topics to consider as you invoke your own strategy, including:
1. What is the difference between DevOps and continuous delivery?
2. What are the key impediments/obstacles to achieving speed/quality in CD?
3. How do we overcome these obstacles?
4. How do we get started on the CD journey?
5. What are the key top-level metrics you want to track in a CD program?
6. What org changes do we need to make to implement CD?
7. What the key capabilities we need to implement?
8. How do we integrate all the different tools we have in our environment?
9. How will the future of continuous delivery evolve?
OK, that last part is a bit more postulation on our part—though we like to think on the practical side, based on the answers to all these other important real-world adoption dynamics.

Tune in and hopefully learn a good deal about where your approach stacks up, and needs to evolve.

As always, you can find said Episode 4 of the Continuous Delivery 101 Podcast by:
- Listening to it on your computer on YouTube
- Downloading it from iTunes store
- Downloading it via the Podcast App by searching for “Continuous Delivery 101 podcast”
Enjoy!

— Scott Edwards
April 21, 2017
An Unanticipated Obstacle to Full Automation

I’ve been concerned about the impact of automation on human employment for a while. My research activity started after November 2016, when I decided I needed to learn whether the much-touted phrase, “We’re bringing back the jobs,” was valid or not.

While it is true that some jobs in the United States have been exported to other countries, a study coming out of Ball State University in Indiana asserts that 88 percent of the jobs lost by U.S. workers is due to automation. Combine this study with the one published in 2013 by Frey and Osborne out of Oxford University that says that 47 percent of the jobs in the United States are risk, then things don’t look so rosy. Martin Ford, in his book, “Rise of the Robots,” goes so far as to say that every occupation is at risk, not just low wage workers who are paid to do repetitive tasks that are easy to learn. Not only will we have machines flipping burgers, they’ll also be perusing volumes of legal documents during discovery processes, which is the type of work that now is done by paralegals and newly hired law school graduates.

Machines are getting smarter all the time and they are getting cheaper. You can buy Baxter, a full fledged factory robot, for $22,000. Baxter can work 24/7 and is not subject to OSHA laws or workers’ compensation coverage.

Software? That’s getting cheaper, too. Every day, another open-source project is made available on GitHub that allows anybody with a computer and connection to the internet to take advantage of the power of artificial intelligence and machine learning. You want Swift code for artificial intelligence? No problem. It’s on GitHub and available to all for free! Want to convert your English ReadMe into Japanese? Simple, just use an API such as Microsoft Translator or Google Cloud Translate.

Given the current trends in technology, it’s not hard to imagine that within the next 15 years to 20 years we’re going to have a world in which fewer people are employed. This future scenario worries me.

About a week ago I was sharing my concerns with Chris Surdak at Surdak and Company. Chris works with large corporations implementing robotic process automation (RPA). He is also on the board of the Institute for Robotic Process Automation and Artificial Intelligence. Chris is at the front lines of using robotics and artificial intelligence to increase commercial efficiency. He and I share a concern about the wave of worker displacement we see on the employment horizon. However, Chris has an interesting take: While he agrees that automation is going to cause significant displace of human workers, he sees an obstacle.

Imagine you are an executive in a corporation and you have 5000 people under you. One day someone like me tells you that you can replace 80 percent of your workers, 4000 people, by streamlining your work processes using robotics and artificial intelligence. You should be ecstatic, right? That is a significant cost savings that is going to directly affect your bottom line.

But, it doesn’t work that way because that is not how many corporate managers think. They want to keep their jobs and in order to do so they need two things: headcount and budget. So, if I come along and tell them that I can create an efficiency that is effectively going to cost them power (less employees and a smaller budget to manage) and maybe their jobs, do you think they are going to jump on board right away? Nope. In fact, they are going to do everything that can to slow down my work until they retire.

Chris’s statement was a revelation to me. Previously, I thought corporations in their continual quest to seek greater profits will do whatever is required to reduce costs and increase efficiency. Most of us who do tech, particularly in DevOps, have embraced this notion. Automation is our friend. We’re lazy. We want the code and machines to do the heavy lifting. And, we’ve created environment where automation is necessary, For example, large-scale container deployment alone went beyond the capabilities of human management a long time ago.

There is a good argument to be made that the factory floor has not had problems using machine automation to improve efficiency. Factories have looked for improved efficiency via tooling since before industrial textile mills used water wheels to power their looms. But, there are other industries that are risk-averse and slow to change; insurance, for instance. The virtue of an industry such as insurance is that it’s ingrained in its culture to manage risk and to be measured when considering change. After all, would you want your retirement annuity to be subject to the whims of a day trader? No. You want your money well-protected by a cautious steward. However, with such safety comes inertia. Thus, there is an intrinsic resistance to change in general and automation in particular. Add in a culture of self preservation on the part of management and … well, you get the picture.

So there are industries that will be resistant—for a while, anyway. Then one of two things will happen. The companies that are not resistant to automation will enjoy the efficiencies the technologies bring. They will gain market share by providing a better product at lower cost and they will trample out the competition—think Uber and the taxi industry. Or, those resistant corporations will realize that to survive, managers who manage by “saving my job” will be given a golden parachute after revealing all the tribal knowledge hidden in their work groups. (Undocumented rules and processes are the bane of process engineers when implementing automation.) Either way, the march of automation will continue and workers will be displaced.

Given the increasing rate at which jobs are being replaced by automation, will these displaced workers find other jobs? This question I leave to you to answer.

— Bob Reselman

April 21, 2017
Self-Healing Service Management and the Future of DevOps
How can DevOps teams take continuous delivery to the next step? Part of the answer lies in self healing, which could optimize performance management and monitoring and underpin the software delivery pipelines of the future.

Self healing means the ability of systems or environments to detect and resolve problems automatically. It eliminates the need for manual human intervention, upon which most systems today still rely to fix issues.

Self healing is better than depending on humans for several reasons:
- Automated solutions are less likely to miss issues by mistake, or to make errors when resolving them.
- Self-healing technology is better positioned to fix problems faster than humans because it can react instantly based on rules or machine learning. It doesn’t have to stop and think about the issue before taking action.
- A self-healing environment can scale without limit. This is a key advantage over manually managed environments, whose scalability is limited in some respects by the availability of human admins.
Self Healing and Continuous Delivery

DevOps teams already take advantage of many other kinds of automated tools to help achieve continuous delivery.

They use continuous integration servers to automate tests. They adopt release automation tools to streamline deployment. They implement orchestrators and scripted infrastructure to provision environments automatically.

Yet, when it comes to performance management and monitoring, the typical organization is a far way off from achieving a fully automated process. Even the best application performance management (APM) tools of today are primarily capable only of detecting problems and displaying information about them. They depend on humans to interpret and react to that information.

That dependency is a major obstacle for achieving fully continuous delivery. You can’t release code quickly if your process for finding and solving problems related to the code is mostly manual.

By making better use of self-healing technology, DevOps teams can speed continuous delivery in the following ways:
- Developers and admins can spend less time on monitoring and more time completing other tasks that can’t be automated.
- Ideally, teams will be freer to switch between development environments and frameworks without having to establish a new monitoring workflow. Self-healing tools will adapt to whichever environment they have to work with.
- Because problems can be found and fixed faster, infrastructure utilization can increase. The need for a buffer between average usage and overload can become smaller because automated healing solutions will be able to fix problems before they lead to failure.
- Teams can establish better continuous feedback loops because, by removing the human element from performance and monitoring, incidents can be reported instantly, consistently and in a way that makes the data easily accessible to everyone on the team.
To be clear, the technology for self healing does not yet fully exist. Developers over the past several years have made great strides in automating configuration, service discovery and environment provisioning, But perhaps it’s time to make them a more prominent focus. They’ll enable the DevOps innovations of the future.

— Chris Tozzi
April 18, 2017
When DevOps Disrupts DevOps
As longtime readers of my blogs will know, one of my hobbies—when not writing about or playing with technology—is modeling. There is a similarity between quality tech work and quality modeling, but the brain power involved in modeling is much less, leaving me time to think about tech issues or listen to audiobooks (when I’m not listening to music, anyway).

In modeling, as in tech, there is a process. The process varies based upon what you’re working on, but is largely the same, with some steps slightly different or skipped altogether and others taking longer. At the moment, I’m building the first day of Gettysburg from the American Civil War, but I simply enjoy modeling—and wargaming with the models, but that’s a separate story for a hobby-focused blog.

The thing is, that process is nailed down, and to the extent it can be, is automated. One step leads reasonably to the next, and while one step is completing on a given model (glue or paint drying, etc.), there is an opportunity to advance a separate project.

Sounds a lot like DevOps, doesn’t it?

The thing is, every once in a while, you will get something that is so different, it totally interferes with the carefully planned flow. My current primary project is a little over 300 figures. While one unit is getting the final two coats of finish (that are basically the same or every model), the next unit is getting primed. Assembly of all the figures came first, so there is only the priming step before painting can commence.

Piper Millin and Lord Lovat cross Pegasus Bridge.

But here is a work-in-progress picture of Pegasus Bridge. This came as a complete set, and took a large chunk of time to get together, find the correct water for, get placed on foundation boards, etc. This is outside the normal process of assemble/paint/base figures and vehicles. The set is beautiful, and even this picture, which was taken well before completion and only to study color and weathering effects versus reality, shows how truly nice the model was. But it totally disrupted the process flow.

DevOps and Multisystem Installations

That’s what complex system installations such as Hadoop or OpenStack do. You’re installing across multiple systems, configuring them for load-sharing and communication, and doing it by hand is a long process that interrupts the normal DevOps flow. Crafting scripts to install them is possible but highly complex and frustrating, simply because of the complexity and interrelationship of the nodes in the system.

Having installed both by hand, and installed both with tools designed to make it easier, I will happily tell you that yes, you will understand more if you do it by hand, and no, that immediate understanding is not necessarily worth the cost of weeks or months figuring it out and getting it installed.

You see, to operate these systems, you will need much of the knowledge you would gain hand-installing, so you’re going to learn it either way. Your choice is a simple one: Get a working system and learn by tweaking settings or swapping nodes in and out, or have no working system and learn by the frustrating process of trying to figure out what you’re supposed to do next, and why that step is failing for your specific hardware.

DevOps and Multisystem Maintenance

The only real problem is that it is painful to maintain these systems. It’s not as simple as
```
yum update [[packagename]]
```
because that would only upgrade one package on one system. But these tools span multiple systems, and even just dependencies generates a massive list for each system in the cluster. Seeing the need, most of the traditional application provisioning tools (Puppet, Ansible, et al) have built or cobbled together solutions that will get the job done. But custom solutions exist for these products, too. The problem is that, like Pegasus Bridge, as beautiful as they are they disrupt the DevOps flow. No system goes without exceptions, and I’ll go out on a limb and say that custom solutions to install and maintain solutions such as Hadoop and OpenStack are worth having yet another tool in the DevOps tool chain. The trend is multipurpose tools to be used on many projects, but the importance of a cloud or big data cluster in your IT operations makes them worth the extra tool. As long as the extra tool reduces the amount of work and the chances for human error (which, in my opinion they do—it is kind of their selling point).

That Leaves Time

These tools (a good example, though in general I won’t be talking about the competing vendors here, is the Mitaka-based release of Mirantis) allow not just installation but also ongoing management of the OpenStack cloud or big data cluster. That means there is less nitty-gritty for DevOps (or operations for that matter) teams to worry about, freeing time for working on what actually gets built on top of these powerful platforms. Cloud and big data both are just enablers for more functionality when looked at in the right light, which means more work (and often more meaningful work) for DevOps teams.

When you consider either of these technologies, check out what’s available for installation and maintenance, even if it is outside your normal DevOps tool chain. Like Pegasus Bridge, the change is likely to disrupt you for a bit, but once you have a tool and process in place, it will just be something that happened—and was going to happen with or without the tools. They just make it less painful.

— Don Macvittie
March 24, 2017
DevOps in Legacy Systems: A Mission Impossible?

The word “legacy” evokes mixed reactions from the IT and business crowds alike. For those who depend on legacy systems such as the zSeries and iSeries platforms, it is a way to deliver stable and reliable applications into production. Business stakeholders who may not know the intricacies of the technology often take a view of, “Why fix it when it ain’t broken?”

On the other hand, for those who haven’t experienced the joys and pains of legacy systems , legacy world will continue to defy their logic and assumptions because their view of the world is shaped by the open system world which has completely different characteristics: being more malleable to change and easy to maintain etc. Hence there is this great divide between the legacy and new worlds. Add Agile and DevOps to the equation with the lofty goals of increased release frequency and reduced lead times, we end up with a divide that is too far to bridge.

The likes of Gartner have come up with constructs such as two-speed delivery to help deal with this legacy challenge. In my view, there is a problem with the idea of dual-speed because it is based on a wrong analogy to start with — that of allowing vehicles to move at different speeds depending on whether you are on the motorway or a city road. That analogy works well for traffic management scenarios but not for complex IT architectures we see in the enterprise. The dependencies across the systems in the enterprise landscape are just too intricate and too tightly coupled to be defined using such a binary logic. In my experience, I find that many Agile and DevOps transformations hit the roadblock and never achieve the desired levels of productivity and effectiveness because of an oversimplified view of these complex dependencies across legacy and open systems.

The three biggest challenges that prevent a legacy DevOps transformation are mindset problems, architectural constraints and tooling limitations.

The Mindset Problem

“There are no legacy systems, just legacy thinking.” That’s a great quote I heard recently from an engineer in a large organization trying to adopt DevOps at enterprise scale. The initial resistance from teams to move into DevOps ways is due to reluctance to change, emerging from an inertia of doing things a certain way for years if not decades. The mindset problem is as much a problem for those outside the legacy world as those inside. The “outsiders”—typically leadership, management team and consultants—tend to oversimplify the problem and, too often, rely on examples from the open systems world. This just alienates the incumbent legacy teams and makes them even more skeptical of DevOps driven change. The outsiders need to have a genuine appreciation of the challenges with legacy architecture and need to work in collaboration with the legacy teams if they really want to transform their organization.

Architectural Constraints

Legacy systems have certain characteristics such as being static, tightly coupled and monolithic blocks of procedural code. These systems were not built for the modern Agile workflows that focus on incremental and iterative deliveries. A significant majority of these systems are written to operate in batch mode.Testing the systems requires cycling through multiple business days, which simply limits the level of agility you want to introduce in the organization. I am currently working with a legacy engineering team in the financial services space where you have to cycle through a minimum of 20 business days to simulate even 70 percent of the test scenarios. However, there are a number of solution options to overcome the architectural challenges; they range from killing applications to migrating to modern platforms or re-engineering legacy code. Not surprisingly, the right approach depends very much on your context.

Tooling

Tooling in the legacy world to support a DevOps ways of delivery has always been a challenge. But it is fast changing now, thanks to a number of startups and even old-school IT product companies investing in this space. If you consider zSeries as an example, there are better IDEs (integrated development environments) such as RDz that can replace traditional code editors typically based on “green screens.” These IDEs provide great features such as better navigation, debugging, split views and searches on the legacy code. One of the CIOs I worked with made it clear to his team that he wanted to get rid of “green” screens from the IT floor. There are tools such as SmartBear Collaborator that support a great peer review experience and help bring one of the original XP practice of pair programming to the legacy world. There are tools such as endeavor from CA that promise automated build and deployments in the z-series world. Similarly, there are tools that support test automation of mainframe applications and tools for code coverage and code quality checks on legacy programming languages like COBOL. XATester looks like a promising tool for unit test automation from a Danish startup called Xact.

While this write-up is not intended to be a compendium of all DevOps and engineering tools, the central message is that there are a number of DevOps tools emerging on the legacy landscape which can help your legacy delivery teams get on to the DevOps and Agile journey.

What is more important is for organizations to realize that many of these legacy systems will remain part of their IT landscape for the next foreseeable future and it is time to invest in these core systems. They may not look as appealing as the digital platforms, but they are fundamental for businesses and you cannot ignore them on your DevOps journey. DevOps in the legacy world is not going to be smooth, but it is certainly not a mission impossible!

— Aditya Vadaganadam

March 17, 2017
The Tester: Saving Your Soul in a DevOps World

Software testers have seen their careers go through some ups and downs in recent years. In the days of Waterfall, the tester’s job was clearly defined. Like other functions of software development and delivery, testers lived in a neat and tidy universe, testing features and design, finding bugs, sending back feedback to developers for correction and eventually, passing off code to the production people to work their magic. Testers had discrete roles with strong value propositions in the software organization.

Then, Agile came along and new teams were created, lumping together developers, QA and Ops people for the sake of better collaboration and efficiencies. This began to create some confusion regarding who was responsible for what, when.

DevOps brought order to the process, adding automation to every step, speeding cycles and reducing bottlenecks in a highly standardized, streamlined approach. In the continuous delivery pipeline, each step of the process creates its own contract for software quality—and that’s good.

Development teams still value high quality, which customers demand. Yet in some DevOps circles, there’s been talk that testers aren’t needed much anymore. They bring bottlenecks to the process, forcing developers to work outside of the DevOps toolsets for code management and testing to respond to a defect ticket posted by the tester in an issue tracking tool.

Eliminating that layer of process helps developers be as efficient as possible using continuous integration (CI) tools, proponents say. As well, some developers are more than comfortable handling testing, using test-driven development (TDD) methodologies.

On the other hand, there’s faulty thinking here. Testers, whether deploying manual or automated testing methods, provide a valuable second set of technical eyes. Testers have a solid take on the user experience perspective, not just the fact that the code executes correctly.

This human behavior and design insight is critical to the ultimate success of a software application. Finally, not all developers want to spend time testing nor have the skills to do so outside of unit testing.

Moving forward into this new age of DevOps, testers play a vital role. The distribution of the tester into each phase of development (rather than at the end) is necessary to maintain both high standards of delivery and to keep communications fluid between different parties.

Here’s how testers can contribute to DevOps teams:

Testers need to be included in the continuous integration process

Developers might be automating their unit test coverage, but they don’t replace all the test activities a tester can bring to the table. There are still other types of functional and non-functional test activities that must occur to validate the health of the product. Instead of transferring more testing responsibilities to the already overworked developer, testers can integrate their processes and systems with those of developers.

Although there are version control tools (GitHub) and feature tracking systems (JIRA) to help developers, testers are often disconnected from these central tools. To connect with developer tools, testers can start creating and versioning their test cases within those development platforms. By doing so, testers can radiate feedback to the development team on issues to fix quickly through source code check-ins and in a way that won’t disrupt the normal flow of work.

If testers can learn to use the DevOps tools developers love, much of the conflict goes away.

Testers should increase their adoption of test automation, no matter their skill level

Testers can learn newer test automation driven frameworks, especially those that help with transitioning manual test cases into automated scripts that can be reused, such as Cucumber or Gauge.

For example, Gauge allows testers to write test specifications in Markdown, which provides a simple transition to automate your favorite programming language. These newer tools help testers transition to test automation despite not having experience writing tests this way—a common issue for testers working in DevOps.

A growing number of test case management providers are starting to integrate their test design and execution around these automation frameworks to make the manual to automation transition smoother.

Although developers might have automated unit test coverage, they struggle with developing functional and user acceptance automated tests. Testers can provide value by suggesting to the development team which are the best areas for test automation based on common issues and their unique knowledge of the application.

They also can recommend areas for exploratory or manual test execution, depending on application complexity. This is especially helpful when teams starting down the path of DevOps do not have a regression test suite built that can be triggered through CI.

Testers should focus on quality over finding defects

Developers often complain that when testers find defects, it slows down their development processes. On a fast-moving Agile or DevOps team, filing bugs in a defect or issue tracking system is a total waste of time and creates unnecessary clutter. Developers want instant feedback on code that’s broken and the related feature, rather than seeing another defect ticket they must open and close upon fixing.

Therefore, testers should be giving feedback directly to the features that are being developed within a branch of code. They can do this through adopting tools that facilitate “ChatOps,” such as HipChat and Slack, which can integrate with version control systems. Testers can also provide direct feedback through a commenting system on the feature itself.

With the advent of Agile and DevOps, which emphasize speed and frequent releases, testing is more important than ever in software development. By making some slight modifications to how they work and the tools they use, experienced testers can add considerable value to these teams by providing another layer of quality checks and a holistic view of an application’s performance and usability.

About the Author / Ryan Yackel

Ryan Yackel is Director of Product Marketing at QASymphony. He is responsible for all marketing activities around product releases, including development of strategic messaging. Ryan also helps to train the Sales teams on key value propositions for new products and features, and he provides customer feedback to the product team for roadmap initiatives. He can be found online at Twitter and LinkedIn.

March 16, 2017