devopsy.kinsta.cloud

Category: DevOps Practice

Why You Can’t Have DevOps Without Test Automation

The rapid adoption of digital technologies by enterprises to stay nimble, competitive and quality-driven has given digital transformation wide currency. The process facilitates decision-making and allows organizations to innovate, accelerate the time to market and be responsive to customer needs.

The world of today is for those enterprises that can quickly turn an idea into a consumer product or service, absorb feedback from the market and customers to refine the product/service and relaunch it. This is where DevOps can help such organizations in terms of offering continuous development, innovation, integration and deployment.

Fixing Glitches with DevOps Testing Strategy

To stay on the right side of customer preference, enterprises must roll out software updates on a trot, which is not easy by any stretch of imagination. This is because every update needs to be tested across devices, operating platforms, frameworks, geographies and networks. If the DevOps approach is not right in executing the process of development, testing and deployment, the inherent glitches can prove to be costly. In many cases, in a bid to be ahead of the curve, enterprises do not give testing its due.

In other words, products or their updates are often launched without letting them pass through a stringent QA pipeline. The consequences of such quick deployment without adequate testing may manifest themselves in various ways. These may include the quality of product taking a beating, regulatory bodies subjecting such organizations to censure and penalties, and customers taking a flight to the competitors. With business reputation at stake, enterprises are well-advised to follow a strong DevOps approach and identify/fix glitches quickly in the pipeline.

DevOps and Test Automation

When it comes to DevOps specialists discussing key elements that go into a continuous integration (CI) or continuous delivery (CD) pipeline, they mostly talk about code repositories or CI servers. Interestingly, DevOps test automation finds less mention in the rarefied circles where strategies are planned. This is despite the DevOps test automation being considered an integral part or medium to achieve outcomes like CI and CD.

DevOps specialists mostly focus on aspects such as integration and deployment of the software, thereby giving test automation a miss. This can be self-defeating and invariably goes against the grain of DevOps philosophy where QA forms an integral part along with development and operations. Does this mean these specialists are more tuned into the manual testing aspect while leaving test automation altogether? The probable answer would be the ingrained belief in some DevOps experts about the efficacy of manual testing. This would not have raised any hackles until the advent of continuous delivery chains.

Why Test Automation for DevOps?

The two outcomes of DevOps, CI and CD, mean the development pipeline should deliver quality products quickly and consistently. Besides, the products should be upgraded periodically based on market feedback. This means products should be tested for glitches early in the SDLC to save cost and time. What better way to achieve the same than by implementing test automation in DevOps QA.

Faster test outcomes: Test automation can be implemented by writing a test script on a tool. The code in development is put through that script and checked for expected outcomes. Any deviation is noted and investigated for possible errors. This is way faster than manual testing, where QA specialists find repetitive testing frustrating and have a tendency to miss errors.

Consistent results: Consistency in QA means the software behaves on expected lines throughout the delivery pipeline. Test automation can validate the code for variables at different stages and mitigate the chances of human errors.

Ease with regression testing: When a major update is implemented, certain types of tests, such as usability testing, are performed manually. However, when there are more variables and the software needs to be checked across devices, operating systems, browsers, geographies and networks, then automatic testing takes the cake. It can work non-stop and check for deviations or non-compliance.

Human resources utilization: During manual testing, scarce human testers are deployed in tasks that require constant attention. Further, to avoid boredom arising out of conducting repeated testing, QA testers can give certain tests a miss. Imagine the type or number of glitches escaping the test dragnet and later ruining the user experience. However, with test automation, repetitive testing can go unhindered wherein scarce human resources can be deployed for better activities.

Conclusion

In today’s age of fast software releases, quality can be ensured through a smooth DevOps-led pipeline ensuring CI and CD. However, achieving consistency in testing and pre-empting issues like cybercrime are underpinned on implementing DevSecOps and enabling a culture of quality control at every step of the way.

April 30, 2020
Why the Public Versus Private Blockchain Debate Is the Wrong Conversation
When blockchain burst onto the scene in 2009/2010, we watched as Bitcoin forever disrupted our monetary system. As the technology hits critical mass—a recent Deloitte survey showed a 2000% increase in interest in blockchain since its inception—the revolution has forked into two sides. There are the purists who want to only keep blockchains public, and the upstarts who see huge value in its private uses.

When lines are drawn in the sand of any disruptive change, it can seem like there is an undeniable right. But, what if we’re getting in the way of real revolution by fighting an unnecessary battle?

Breaking Down the Beef

The Argument for Public, or Permissionless, Blockchains

Down with centralization, eliminate the middleman. Satoshi minimalists evangelize for a pure peer-to-peer system, uncorrupted by larger networks, corporations or governments. The beauty of Bitcoin was the emergence of databases both immutable and unburdened by firewalls, accessible to anonymous players who bought into a shared economic belief system. Complete transparency on a global scale was its defining feature. In this thrilling first wave of ICO-driven public blockchains, it was considered unsportsmanlike, if not unethical, to build and sell permissionless entities without an associated cryptocurrency. Any enterprise-driven motive would surely pollute the blockchain’s democratic roots.

The Argument for Private, or Permission, Blockchains

Amid all the enthusiasm for open read-and-write blockchains around cryptocurrencies, there started a rumble around the question of private blockchains. Permission-based platforms, in which access to read and write was assigned, were designed to address two vital issues:
- Data relevancy — Bitcoin lovers champion a world in which institutions don’t have control over information and transactions. But, the reality in the enterprise world is commerce businesses need a hold on both. If the data you’re tasked with managing is relevant to only a handful of parties, why would you want to broadcast it to more?
- Data privacy — If at the end of the day, not all data is meant to be shared at large—supplier-vendor relationships, financial transactions or health data, for instance—it has no place on a public blockchain to begin with.
What the Debate Is Missing

The conversation regarding public versus private blockchain doesn’t have to be a polarizing one. It’s not an either/or debate but rather a question of application. Private blockchains don’t have to be viewed as the enemy, or a replacement for public ones. They are simply a case-specific option.

When taken out of the theoretical arena, there is room for both open read-and-write blockchains and those with access restrictions. What we find in practice, having developed numerous blockchain applications for both entrepreneurs and intrapreneurs, is that the apparently different requirements of each tend to converge over time.

That is, many applications built by entrepreneurs will integrate with one or more large corporate enterprises at some point, and will therefore need to address their needs (affordability, speed, boundaries, etc.). Similarly, many enterprise applications are tackling obstacles that currently prevent them from making their solutions more open and capable of incorporating tokens of some form.

Both sides are invested in the value of bringing integrity around data. So, arguing over whether to support private or public blockchains misses the point. The real world has needs for both.

A Call for Both Public and Private

The gifts of blockchain technology are extraordinary. It is our best defense against the threats of theft, fraud and hackers, and can bring needed efficiency to every aspect of the digital landscape. Enterprises need to ensure some level of security, privacy, compliance and performance that only a private blockchain can provide.

We also need public blockchains. They serve an enormous purpose whose value should never be denied. But private blockchains are not the enemy. To waste time demonizing them gets in the way of our tackling larger, more urgent questions. Where is this technology going next? How can we make best use of it in our daily lives? And, most importantly, how powerful will we be if we understand we’re all in this together?

This article was co-authored by Andrew “Flip” Filipowski, co-founder and co-CEO at Fluree. Flip was COO of Cullinet and was also the founder and CEO of PLATINUM technology, inc. He grew PLATINUM into the 8th largest software company in the world at the time of its sale to Computer Associates for $4 billion dollars. Upside Magazine named him one of the Top 100 Most Influential People in Information Technology. A recipient of Entrepreneur of the Year Awards from both Ernst & Young and Merrill Lynch, Flip has also been awarded the Young President’s Organization Legacy Award and the Anti-Defamation League’s Torch of Liberty award for his work fighting hate on the Internet.
April 29, 2020
DevOps Practices: How to Make Valuable Changes to Your Teams
DevOps has become a buzzword within the software development industry, promising rapid turnaround times for requested changes. As for developers, DevOps promises to increase productivity while reducing the risk of production failures. The book “The Unicorn Project” by Gene Kim describes this process for developers as The Five Ideals.

In this article, we’ll focus on only two of them:
- The Third Ideal — Improvement of daily work
- The Fourth Ideal — Psychological safety
Psychological safety and improvement of daily work make it easier for developers to create changes for their teams, and the work to implement these ideals is not difficult. From automated testing to production monitoring, there are many great things that even junior developers can help bring to their teams that will help their entire team make safer changes to their code.

So, what are these easy changes that can be made? Below is a list of tasks that can be done to improve the projects you are assigned to, with most taking less than four hours to complete.
1. Set up a continuous integration server for your project, such as Jenkins or Team City. This will allow for automated jobs to be run for your code.
2. Create jobs to run testing for pull requests as well as testing jobs to run overnight. This will show the current state of your project’s tests on a nightly basis and will allow for the number of failing tests to be monitored over time.
3. Add a code coverage tool to your automated tests. This will show what is being tested in your code and what is not being tested, creating opportunities to help shore up your teams testing suite as well as confirm that your changes will be covered by existing tests.
4. Create an automated deployment job on your CI server. This will convert the manual deployment process in a one click job, allowing the code to be deployed on command.
5. Modify your project to extract metrics of the project, allowing your code to easily be monitored for faults in production.
The tasks highlighted above will quickly add value to your team while helping protect you for making dangerous changes to your team’s code base. I recently changed teams and have moved to a new web project. The project runs fine on my local, but it is hard to track what is affected by my changes. The team already has a CI server to build their code, but there is no point in setting up any testing jobs as the project has fewer tests than I have fingers.

Instead, I created a monitoring tool that captures all requests to the website and records their results. With the tooling in place, my team can now see how long it takes for each part of the website to load, as well as how many errors are being produced. Now, when I make any changes, I can check these metrics to see if my change is causing issues to the site, increasing my confidence in my code.

If you are interested in learning more about DevOps and how to take advantage of it, here are a few good reading resources to look into:
- “The Unicorn Project,” by Gene Kim.
- “The Phoenix Project,” by Gene Kim, Kevin Behr and George Spafford.
- “The DevOps Handbook: How to Create World-Class Agility, Reliability, and Security in Technology Organizations,” by Gene Kim, Patrick Debois, Jez Humble and John Willis.
April 28, 2020
5 Ways Your Business Can Benefit From DataOps
The amount of data that businesses generate and collect continues to soar, and enterprises that want to accelerate their end-to-end processes and obtain business insight can’t keep using the manual data management processes they have relied on for decades. And the situation will only get worse in the coming years. IDC expects the amount of data to grow at a 32% CAGR to 180 zettabytes by the year 2025. Fortunately, DataOps can help.

DataOps is a relatively new and still-maturing discipline that arose about five years ago. It’s built on the foundations created by agile development and DevOps, and the goal is to bring similar principles to data analytics and data science to improve data quality and reduce the time required for obtaining actionable business intelligence.

Over the next few years, enterprises that resist adopting DataOps will spend even more time reacting to data errors and broken manual processes, and fall even further behind in their ability to provide timely, accurate information to business leaders. Meanwhile, those that embrace DataOps will create streamlined, automated data pipelines that enable data administrators and data scientists to optimize business processes, focus on higher-value tasks and support decision makers with the best possible intelligence.

In this article, we’ll review the principles and power of DataOps and five specific ways every organization can benefit from it.

What Is DataOps?

Because of the massive size of datasets, performing data analytics today requires automation to be able to do validity testing, analyze the activity and behavior of the data pipeline itself, and detect anomalies and outliers that can indicate a quality issue with the data or the pipeline. Based on the efficacy of agile development and DevOps principles to deliver high-quality results orders of magnitude faster than previous approaches, relevant principles and best practices were borrowed and codified in the 18 principles of the DataOps Manifesto to enable the same types of automation and quality-detection capabilities within the data pipeline.

A foundational concept of DataOps is that “analytics is code,” which means that everything–collection routines, ETL routines, even the analysis routines that drive metadata and business intelligence-level summaries–needs to be modular, automated and easily and instantly repeatable. As businesses work toward this goal, operational benefits will accrue, such as the ability to receive real-time alerts on streaming data, so quality issues can be resolved before bad data gets propagated or has a chance to impact decision making.

The Power of DataOps

DataOps is already enabling businesses to transform their data management and data analytics processes. For example, like DevOps, DataOps lets teams easily spin up isolated, safe and disposable testing environments that allow them to experiment and innovate (Principle 12 of the Manifesto). However, while developers typically focus on applications with small test databases, data analysts and scientists may need to spin up a sandbox environment that includes applications along with terabytes or even hundreds of terabytes of data. By easily implementing intelligent DataOps strategies such as automation, cloning, predictive analytics and more, spinning up massive disposable data environments becomes possible.

DataOps principles are also enabling businesses to act on their massive production datasets in ways that were unimaginable just a few years ago. For example, DreamWorks can now easily share the datasets of its films in development with teams of creative artists around the world, enabling rapid collaboration and dramatically shortening production times. Another example is the genomics company, WuXi NextCODE, which has developed a genome platform that can compare human DNA–millions of bits of data–and integrate the data on the fly to explore the differences or mutations that may cause cancer or rare diseases.

There are also many financial services companies, in America, EMEA and APAC, all leveraging NetApp for moving to a hybrid cloud data pipeline model. The hybrid model allows them to maintain compliance for protected and sensitive data, while taking advantage of cloud cost savings for non-sensitive data and applicaation components. This allows for a strategy that delivers both regulatory compliance and the flexibility of a hybrid data pipeline architecture.

However, you don’t have to be a film studio, genomics company or financial services firm to benefit from DataOps. Every company that needs to obtain timely actionable business intelligence will benefit in five critical ways.

Top 5 Benefits of DataOps for Any Business
1. Reducing toil: Similar to DevOps, DataOps is fundamentally about process-oriented methodologies and automation, which dramatically increases the efficiency of workers. By baking intelligent testing and observation mechanisms into the analytics pipeline, teams can stay focused on strategic tasks instead of poring over spreadsheets looking for anomalies.
2. Better-quality data: Creating automated, repeatable processes, along with automatic code checks and controlled rollouts, reduces the chance that any type of human error will get distributed to multiple servers and take down the network or produce erroneous results.
3. Faster access to actionable intelligence: Reducing toil and improving data quality leads directly to faster access to actionable business intelligence. Automated ingestion, processing and summary analytics on incoming data streams, combined with the elimination of errors, can deliver insights into customer behavior patterns, market shifts, price fluctuations, etc.–instantly instead of hours, days or even weeks later.
4. Seeing a bigger picture of dataflow: Beyond the business-critical day-to-day insights, DataOps can provide an aggregated view over time of the entire dataflow, across the organization and out to end users. This can reveal macro trends, such as adoption rates of features or services, or search pattern deltas over time. Even behavioral or geographic patterns for focused or global data sets. Creating such a view would never be possible for teams that are constantly reacting to anomalies and errors with manual processes.
5. Career enhancement: Data analytics and operations professionals who learn how to implement and manage DataOps processes will enjoy career benefits as they become the leaders of the next generation of data teams and set the standard for data practices for at least the next 10 years. The business also benefits from increased employee satisfaction and retention as repetitive and monotonous processes are replaced by an exciting and fast-moving organization focused on innovation.
Over the next five years, DataOps will become mainstream in the same way that DevOps did. The benefits are too compelling, and the consequences of ignoring it are too dire. However, as companies progress on the DataOps journey and find success in leveraging its principles to drive business intelligence and streamline processes involving large datasets, they will eventually confront the limitation of their infrastructures. As a result, they will need trusted technology partners that can help them ensure data replication, data distribution and data availability at a scale that dwarf’s today’s requirements.
April 28, 2020
Red Hat Extends Scope of Ansible Tools

Red Hat today announced it has extended the analytics capabilities it makes available for instances of the Ansible IT automation platform and has made the Ansible Automation Services Catalog now generally available.

In addition, Red Hat has increased the number of certified playbooks it makes available from both the company and third-party partners within the Ansible Content Collections. That repository now houses more than 1,200 certified Ansible Modules.

Announced prior to the Red Hat Summit, which is taking place online this week, these offerings are part of a series of cloud services the company has developed to foster the adoption of Ansible to automate a wide range of processes.

Tom Anderson, vice president for the Red Hat Ansible Automation Platform at Red Hat, said that as Ansible is employed to automate more IT processes, more IT teams want to better understand how the playbooks being created using Ansible are being used within their organizations. The latest updates to the Ansible Analytics module add the ability to filter analysis by Red Hat Ansible Tower cluster, by date and between organizations. IT teams can also discover job statistics by total runs, total time, average time, success rate, most failed tasks and the failure percentage of that task.

At the same time, the Ansible Services Catalog makes it easier to access curated playbooks that have been vetted thoroughly, he said.

Anderson noted Ansible continues to gain traction because it provides a declarative approach to automating IT processes that can span multiple platforms. Interest in automation has increased substantially in the last several years. The challenge that many organizations now face is that as various providers of platforms have made IT automation tools available, organizations now are managing multiple islands of automation, he said, adding Ansible provides a means to unify IT automation under a common open source framework.

The pace at which organizations have been automating the management of IT is expected to increase substantially in the weeks and months ahead. In the wake of the economic downturn brought on by the COVID-19 pandemic, organizations that are reducing IT staff will look to automate processes to enable the remaining IT employees to manage more efficiently IT environments that continue to grow in size and complexity.

That increased reliance on automation should also advance adoption of best DevOps practices, as more IT operations teams embrace managing IT infrastructure as code.

Ansible, of course, is not the only platform capable of automating IT processes. However, because of its reliance on a declarative approach, Ansible is more accessible to IT operations staff who lack programming skills. In theory, DevOps teams are increasingly taking control over the management of IT infrastructure using programmatic tools. However, there are still many IT organizations that prefer to have access to IT infrastructure centrally managed by IT operations teams that typically make resources available to developers via a self-service portal. However, while there may be a debate over who will automate IT processes, the one thing everyone does agree on is the time to do so is now more than ever.

April 27, 2020
A Prescription to End the Myopia of Software Metrics

“People don’t want a quarter-inch drill. They want a quarter-inch hole!,” Theodore Levitt, Harvard Business Review.

As someone who has purchased many drills in my life—and drilled my share of holes—this quote has always struck me as a bit, well, myopic.

Don’t get me wrong, I’m not arguing with the sentiment behind the quote—that organizations need to focus on customer needs versus the products they make—but perhaps the example could be updated to reflect our times.

Part of the issue I have with this quote is that, unless you only want a peep hole, a quarter-inch hole is not a solution in and of itself. No one needs a quarter-inch hole in isolation.

You are most likely working on a bigger project that requires multiple holes. You have an outcome in mind, that outcome requires some holes drilled and a drill is the best way to make holes. The holes are an intermediary step.

For example, the last time I purchased a cordless drill was because I was remodeling my kitchen and my existing drill had reached the end of its useful life. Of course, I needed to drill holes in the process, but that was not my ultimate objective—the kitchen was.

By now you are probably asking yourself, “What does a 60 year old quote about marketing myopia have to do with DevOps and software development?”

I would argue that the same principle applies to the metrics we apply to software development. Using myopic measures of success can prevent us from meeting the needs of customers, or of our own organizations.

In a previous post, I wrote about a Forrester Report that urged CEOs to end the madness of software metrics and called out software organizations for using arcane, indirect measures of success.

Some of the metrics we use to measure DevOps success today are the equivalent of measuring the number of holes we have drilled, the speed at which we drilled them or the quality of the holes. While these are certainly important to gauge, they do not correlate directly to whether we are meeting the needs of our customers, or of our own organization.

As a follow-on to the report referenced above, Forrester conducted a survey of over 300 senior business and software leaders, which uncovered some interesting data points.

Eighty-six percent of respondents said it’s important to align software delivery practices with the overall objectives of the business. And 80% said measuring the overall value of software initiatives is important or critical. Makes sense.

Yet, very few are actually tracking business-oriented metrics that align to those goals. Only 38% are correlating software initiatives to their impact on revenue, and fewer (37%) are attempting to measure ROI for software development. Impact on customer experience was slightly better, but still measured by less than half of respondents (45%). These are metrics most business leaders care about—but few software organizations are tracking them.

The study highlights the metric madness—or myopia—that causes disconnects between software development teams and business leadership. BizOps is a new framework that is gaining traction in the industry because it promises to eliminate this disconnect.

At its heart, BizOps is a framework for optimizing software development and delivery to foster the achievement of business objectives. Essentially, BizOps helps connect the dots between traditional measures of software progress and metrics of business success. It’s what we need to end the myopia of software metrics.

April 27, 2020
6 Common Pitfalls to Avoid While Building Out Your AI Tools

Enterprise-level companies, with thousands of employees across the globe, can have their AI framework replicated by B2B companies of nearly any size. In fact, even without access to a ton of proprietary data and immense computational power, you can build out effective AI tools.

To begin with, you can introduce publicly available datasets from movie review websites, e-commerce review sites and elsewhere to train your system. When doing so, be sure to double check the license details of such publicly available corpuses, as some will not allow commercial usage. Then, during the preprocessing stage—after removing swear words, jargon and other irrelevant elements from your corpus—you can add noise and slightly alter your existing data to create even more relevant data points in your corpus.

After this, you can build a semi-supervised learning algorithm through techniques such as tri-training. In tri-training, we essentially train three different neural networks over binary classification data. Once two models agree with one another, the data is introduced to the third network and then re-trained until all three models are in agreement. However, before your AI tools are up and running, it’s important to be mindful of the following common pitfalls.

Overfitting AI Models

It may seem counterintuitive, but you really do not want your AI model to be too accurate. If your cross validation score is anything above 100, then you might need to reconsider deploying the model, since it might be prone to overfitting. If your training data is overfit, one small change can render the entire model useless. Your model should be accurate. However, it cannot be too accurate, as it needs to be adaptive.

Class Imbalance

When training your AI tools, it sometimes makes sense to include a disproportionate amount of your target variable. For example, perhaps you’re training your computer vision AI tool to recognize planks of wood in a warehouse. Even if planks of wood only account for 1% of your total inventory, you’ll want to oversample the minority class. In this case, you’ll likely want 60% of your sample to be planks of woods. You really want to amplify this number to ensure you have the right class balance.

Data Leakage

Another common misstep is the failure to capture data leakage, which can occur in any number of ways. A common instance occurs when an element used to test the model slips into the training data. This generally causes your model to appear to be performing well. However, its predictions might not be as good when deployed in production. To rectify this, it is vital that you keep track of any changes to your AI tools. In addition to your organization’s code reviews and configuration change reviews, you should have data set reviews to ensure your AI engineering is being done properly. Any parameter change within your AI tools needs to be fastidiously documented.

Concept Drift

All AI tools are dependent upon the shape of the data that comes into your system. For example, perhaps your AI model was trained for response times being between 1-100 seconds, and all of a sudden the response times are between 1-1,000 seconds. In this case, your model will likely fail to capture the new data points. It’s important to be aware of such an occurrence, so that you can retrain your model accordingly.

There are many techniques that can be used to identify instances of concept drift. Rather than waiting for feedback from your customers, you can set up anomaly detection in a way that it identifies concept drift on incoming data and automatically updates your AI model.

Using Personal Data

With the CCPA and GDPR in place and a federally mandated U.S. data privacy law on the horizon, organizations are wary of using their customers’ data to train their AI tools. It’s prudent to train your AI system with anonymized data; this way, you won’t set yourself up for problems down the road.

Failing to Ensure All Your AI Decisions Are Explainable

Most AI-enabled decisions are enacted upon by teams rather than individuals. Hence, it’s vital that your AI model’s recommended courses of action come with explanations. Additionally, these explanations should have confidence intervals, making it easier to interpret results.

During root cause analysis and remediation, it’s often unclear which team is ultimately responsible for the issue at hand. If the AI’s suggested course of action comes with an explanation, this helps immensely during the remediation process. For example, if the starter event is a network build-up, the issue likely lies with the engineering team. Alternatively, if the starter event is a network configuration change, the network team is the likely culprit. By closely monitoring your AI tools’ performance and ensuring that all AI decisions are explainable, you can quickly solve issues and course correct.

April 24, 2020
3 Ways DevOps Can Increase the Value of Organizational Data

There’s a reason the term “data mining” came into existence. Enterprises sit on a vast resource of information—millions of digital gems that, if fully extracted, can not only answer many of their toughest questions, but also help them meet their most ambitious organizational goals.

DevOps professionals have a unique opportunity to impact this mining process. Their vision can transform the way their organizations utilize data by building fresh capabilities and strategies into the applications and platforms they create. Yet, recent advancements in data science, particularly the use of parallel processing platforms that interrogate massive datasets in near-real time, are changing the game as well.

Time-to-Insight Is Shrinking

Parallel processing improves analytics along two important paths.

First is the time requirement. When organizations are able to process billions of rows of data in milliseconds, it changes the value of that data. In many industries—financial services, for example—there is a big difference between a query that is answered in seconds or minutes, versus one that is answered in a fraction of a second. Those small deltas in time can make a big difference in the bottom line. As they say, time is money.

The other path is the curiosity factor. In conventional analytics, the paradigm is to ask a question and get an answer. But if the user can ask one question, get the answer and then ask three more questions based on that result, without concern for a time penalty, it creates an opportunity to be more curious. This is a pain point that is underappreciated and largely overlooked in most organizations. Without speed and agility, decisionmakers can end up in a rut, asking the same questions over and over again because the time/value equation never improves.

When queries only take milliseconds to answer, it’s easy to ask new questions, dig deeper into data and look at problems in entirely new ways. DevOps has the ability to leverage the potential of parallel processing to empower their organizations and significantly raise the value of data.

Here are three of the more prominent ways.

Use Multiple Processors to Scale Processes

A central data repository facilitates analytics throughout the organization, especially for enterprises that make heavy use of business intelligence (BI). While a centralized location is advantageous, it also places huge demands on DevOps to deliver adequate performance.

Parallel processing can support this approach, even in mixed GPU/CPU environments. Parallelized code allows a processor to compute multiple data items simultaneously. This is necessary to achieve optimal performance on GPUs, which contain thousands of execution units. However, optimizing hybrid execution also translates well to CPUs, which increasingly have wide execution units capable of processing multiple data items at once. Parallelized computation can improve query performance in CPU-only systems as well.

Accelerated analytics platforms allow single queries to span more than one physical host when data is too large to fit on a single machine. Such platforms can scale up to tens, even hundreds of billions of rows on a single node and can also run across multiple nodes. Load times improve linearly with the number of nodes because loading can be done concurrently.

Connect Data Silos for Deeper Understanding

New insights arise when questions are asked not simply within one silo or another, but at the place where those silos intersect and relate to each other. No longer do decision makers have to look under one rock and then move on to five other rocks; instead they can treat their organization’s data stores as one complete, interconnected resource.

The advantages of connected silos don’t stop there. Using accelerated analytics, it’s possible to combine internal datasets with external sources. Analysts may want to review the impact of demographic shifts on retail performance, for instance, or of weather on public safety.

Use Advanced Analytics to Tighten Datacenter Security

No one has to tell developers that cybersecurity is essential. Yet, the difficulty often lies in staying on top of the challenge—log files, for instance, can grow to billions of rows of data every day.

The ability to pull security data into an accelerated analytics environment provides instant insight into data center security. Dashboards that incorporate such capabilities allow staff to interrogate log files and other security datasets in near-real time. Furthermore, spatiotemporal analysis—the ability to combine and analyze time and location data—is becoming increasingly valuable for intrusion forensics. When professionals understand what region of the world an attack is coming from, it becomes that much easier to minimize or stop altogether.

Giving Data a Second Look

Despite the incredible rise in big data, organizations don’t always turn to their repositories to answer their hardest questions. They assume the platforms they have, or the data they possess, are too unwieldy, slow or difficult to leverage.

Organizations need to go back to the era in which there was confidence in the ability of data to answer those really hard questions. Too many teams have succumbed to asking only those questions they know data can answer, rather than asking harder questions that more advanced analytics solutions can uncover.

With the right technologies and processes in place, the questions that seemed to be impossible are no longer insurmountable. The data that resides in the organization is more capable than anyone thinks. All one has to do is have the daring to ask.

April 24, 2020
Software Testing: The Comeback Kid of the 2020s

The past 15 years have been rough on software testers. The rise and fall of outsourcing, Agile growing from a manifesto to a movement and ever-shrinking release cycles have all taken a toll. By 2011, Albert Savoia walked on stage at the Google Test Automation Conference (GTAC) dressed as the Grim Reaper and declared testing dead. Since then, the emphasis on the “automation” part of “test automation” has fueled demand for Software Development Engineers in Test (SDETs) and left many testers wondering if they must learn programming to save their jobs.

If you look at how frequently people are searching for “software testing” on Google, the trends seem to suggest that the profession is, if not dead, at least flatlining.

However, I think the times are a-changin’. The pendulum will be swinging back—to the point where software testing could be the comeback kid of the 2020s. Sounds crazy, I know. But consider some developments that could turn things around.

Developers Don’t Want to Bear the Testing Burden

On Agile development teams, the party line is that unit testing is essential. It’s the most efficient way to expose implementation errors as they’re introduced. You can feasibly cover each function/method when you test at this level, plus when a test fails, you know exactly what function is broken. Moreover, having a suite of effective unit tests establishes a safety net that alerts the team whenever changes break previously-verified functionality. Those are just a few of the reasons why unit testing forms the foundation of the Agile test pyramid.

But, if you talk to developers in a casual setting, the truth starts to come out. Unit testing requires a lot of work, and they’re skeptical about whether it pays off. Just skim through a few of the 2.5 billion results returned by a Google search for “Is unit testing really worth it?” to get an idea of some of the issues.

Unfortunately, a seemingly endless amount of work is required to ensure that unit testing delivers long-term benefits as a regression testing safety net. Developers are committed to progression testing, not regression testing. Although developers on Agile teams usually write unit tests to check new functionality, few keep them up to date as the code base evolves. As the team moves on to another sprint, these tests become regression tests. Then, little by little, they start failing—but developers have moved on. The number of working unit tests at the base of the Agile test pyramid erodes, as does the level of confidence the test suite once provided.

In fact, data from our internal studies on financial instructions across EMEA shows that as an iteration ends, unit tests cover about 70% of the new code. After a few iterations of code getting extended, refactored and repaired, the original unit tests provide only about 50% coverage of that original functionality. This drops to 35% after several more iterations, and it typically degrades to 25% after six months.

Is Unit Testing Doomed?

Ultimately, developers don’t have the time or desire to keep these tests current over the long term. Unit testing has been a best practice for more than 20 years, yet despite waves of unit test automation tools (including one created by Albert Savoia not long before he declared testing dead), unit testing remains a thorn in developers’ sides. Does that mean we give up the benefits of unit testing altogether? Not necessarily.

In order to take on unit testing per se, testers would need to understand the developers’ code as well as write their own code. That’s not going to happen. But, you could have testers compensate for lost unit test coverage through resilient tests they can create and control. Professional testers recognize that designing and maintaining tests is their primary job and that they are ultimately evaluated by the success and effectiveness of the test suite. Let’s be honest, who’s more likely to keep tests current, the developers who are pressured to deliver more code faster, or the testers who are rewarded for finding major issues (or blamed for overlooking them)?

Another option is to consider new testing approaches that achieve unit testing benefits for today’s most popular development architectures—but without requiring knowledge of the code. Enter microservice contract testing.

Microservice Contract Testing: The New Unit Testing

Interest in microservices (the blue line on the graph below) has been rising rapidly since 2015. In fact, it seems to be on the exact opposite path from software testing (red line).

This rise is hardly surprising. From the architecture perspective, there are significant benefits to microservices. Initial forays into service-oriented architectures (SOA) were a huge step forward in enabling reuse and flexibility. Business logic formerly implemented in monolithic applications was freed and released as isolated independent services, which were loosely-coupled and highly-distributed.

Microservices take the decoupling and granularity to new levels. Business logic is now implemented in the smallest, most discrete units possible. Functionality that would have once been a single service is now distributed across tens, or potentially hundreds, of microservices, each of which can be independently developed and deployed.

In addition to enabling unprecedented agility and scalability, this extreme decoupling opens up promising new options for testing. With the initial waves of services and APIs, API tests had little choice but to pass through a fairly broad set of business logic, which was likely implemented through multiple units of code (e.g., functions).

If an API test failed, there was no clear pointer to what specific code function needed to be fixed. That required unit testing and/or additional debugging.

Now, with each microservice implementing an extremely focused unit of business logic, you know exactly what code is broken when an API test fails. Why spend the time and effort on unit testing if you’re already getting the benefits from something that’s considerably less painful?

You might be thinking, “Testing hundreds of independently-evolving microservices is just going to present a different pain.” Correct. Each integration point is a potential point of failure, and testing all the possible integrations is a logistical nightmare—so don’t. Instead of testing how each microservice interacts with all the other microservices it touches, perform contract testing instead.

Very basically, contract testing helps both microservice consumers and providers ensure that they’re meeting expectations in terms of message format and data. The microservice consumer adds a contract that specifies expected request and response pairs for its various interactions with other microservices. Each contract is used to create a virtual service that the consumer can then test against. If the consumer’s changes result in incorrect/unacceptable request messages, they’ll be alerted immediately by a contract failure message triggered from the virtual services.

On the flip side, the microservice provider can use the same consumer-specified contracts to ensure that their own changes don’t break the functionality which consumers depend on. If the actual provider response suddenly fails to meet consumer expectations, they, too, get a rather clear and telling contract failure message.

The beauty of this approach—even beyond the fact that neither consumer nor provider ever has to stand up the whole spider web of interconnected and independently-evolving microservices—is that you get very specific unit-level failure information without burdening developers. As soon as testing moves away from the code level to the API level, it doesn’t need developers doing it. You certainly don’t need developers to deal with the maintenance, and you really don’t even need them to set up the test. Instead, the task can be delegated to people who view testing—not development—as their profession. We call them testers.

Looking Beyond Unit Testing Is Essential for Ensuring a Positive User Experience

Nevertheless, hyperfocusing solely on unit testing of any flavor can leave you blind to many critical issues. For example: Does every step of the end-to-end transaction work as expected—from whatever mobile or browser interface the user happens to have, through the backend (which might interact with custom/legacy applications, packaged applications, mainframes, databases, third-party services, etc.) and back to the user? Will these same transactions perform acceptably under a sudden increase in demand? If the user exercises the application in ways that the developers didn’t anticipate, will the application respond in a reasonable manner, or at least fail gracefully?

The answers to such questions lie beyond the horizon of the testing performed by the typical Agile team, which rarely extends beyond unit testing and a little integration testing.

However, as organizations invest millions in digital transformation initiatives, this doesn’t cut it. Maximizing that investment requires a flawless user experience from start to finish, 24/7. Professional testers thrive on exposing the glitches that developers tend to overlook but that would cause a CIO to cringe.

I’m confident that software testing will surge in importance in the 2020s if we commit to expanding its scope and effectiveness, which will ultimately elevate its business value. As the pendulum swings back, testers can’t expect to just hang on for the ride. Willingness to jump on the momentum of this decade’s trends—for example, stepping in to solidify the foundation of the Agile test pyramid and embracing new practices such as microservice contract testing—will be essential for navigating the turnaround.

April 24, 2020
How SDM’s Holistic Approach Connects Organizations and Streamlines Delivery Processes
Since Jez Humble and David Farley popularized the term in 2010, continuous delivery has revolutionized software delivery practices.

Creating mature continuous integration/continuous delivery (CI/CD) pipelines has helped enterprises deliver software that performs better, based on a long list of metrics. Failure rates have dropped from 50% to under 15%, and cycle times are measured in hours, not weeks. Organizations get to market faster and release products into production with fewer bugs.

But, there’s a limit to what CI/CD can provide. CI/CD connects only the parts of the software lifecycle that happen inside DevOps. It doesn’t provide insights into where the delivery cycle is breaking down, whether the software is helping the business meet its internal key performance indicators (KPIs), how customers are responding to it or whether last week’s release is achieving the internal efficiencies the business was projecting.

A More Holistic Approach

This is where a more holistic approach to software delivery can help. As we discussed in a previous blog, there’s a move afoot to recognize software delivery as more of a core business process and connect all the aspects of this process in one new, interconnected discipline.

The new discipline, called software delivery management (SDM), changes the game by going beyond CI/CD and empowering developers and other stakeholders to best collaborate and creatively solve problems. It gives them the data, insights and best practices that allow them to truly continuously improve, keep value flowing and efficiently develop and deliver the right products and features to customers.

To follow SDM principles, a management solution needs to rest upon four key pillars: common data, universal insights, common connected processes and having all functions collaborating.
- Data: All information within and around software delivery activity would need to be captured and stored with a consistent domain model to enable and facilitate connected processes, shared insights and collaboration.
- Insights: Visibility and insights would need to enable a common understanding and continuous learning from data across, up-and-down and throughout all functions in the organization.
- Processes: Processes would need to orchestrate the entirety of the software delivery value stream and connect functions together to efficiently bring ideas to market with maximum value and adoption.
- Collaboration: All functions and teams within and around the software delivery organization would consequently work together to amplify value creation efforts.
This holistic approach helps organizations in a number of ways. Here are three benefits DevOps managers can net in terms of the way they work, collaborate with development teams and connect with peers throughout their organizations.

Faster Development

The current process can be slow and tedious. Developers use tools configured to perform one task, and getting a complete end-to-end view across the toolchain is difficult. An SDM solution stores all the artifacts and data from disparate tools that move software from idea to deployment in a unified common data layer. The information is connected and available for those who need it, when they need it and in a form they can use. Just forging these connections will save hours of time looking for data, checking statuses and tracking issues throughout the delivery process.

Better Software

If stakeholders have no end-to-end view of the development and delivery process, they have no way of knowing if the software is meeting the organization’s business objectives. With SDM, the continuous feedback loop established with CI/CD and DevOps is extended to fully include all stakeholders from the ideation phase through user adoption and back to ideation. Better insights are tracked more closely and shared more widely, helping developers do a better job of delivering what the users want.

Better Collaboration

Developers and business leaders don’t collaborate well for a number of reasons. Developers often don’t have a good feel for how a product is going to be used, and business leaders usually don’t have insight into what’s in development or when it will be delivered. This is complicated by scattershot communications processes–where both sides have to sort through spreadsheets, connect over email threads and wait for meetings to move processes forward. With SDM, the silos between the software organization and other stakeholders are broken down, so both sides can share information, communicate more directly and drive toward solutions.

In an upcoming blog, we’ll explore other examples of SDM in action and dive into the business value it creates for organizations.
April 23, 2020