devopsy.kinsta.cloud

Author: Tony Bradley

TMI: BigPanda Separates the Signal from the Noise

Organizations have struggled for years with the problem of too much information. IT professionals can only address so many issues at one time, but many are faced with an overwhelming volume of alerts to try and review and prioritize. BigPanda thinks it can solve that problem with its event correlation platform.

If everything is an alert, then nothing is an alert. There has to be some way to separate issues and alerts that really matter so IT teams can prioritize efforts accordingly and address the concerns with the greatest business or security impact. At the same time, though, it’s crucial to minimize or eliminate both false positives and false negatives.

I spoke with Assaf Resnick, CEO of BigPanda, about the company’s approach to event correlation and why he believes organizations need what his company has to offer. Resnick described the current state and trends in IT infrastructure and IT development, noting DevOps principles and tools have transformed the entire data landscape.

Customers, he said, are asking themselves, “How do I move away from this monolithic application hierarchy that deploys once a quarter?” The goal, according to Resnick, is not just more reliable uptime, but delivering speed and agility at the same time.

The BigPanda platform doesn’t try to reinvent the wheel, per se. In fact, it more or less relies on users having other wheels in place. BigPanda sits on top of what a company already has and pulls in data from other platforms such as New Relic or Splunk. It then normalizes and correlates that data so it can spit out results on a scale that mere mortals can comprehend. “Instead of 20,000 alerts from 15 different monitoring tools, I may only have 36 meaningful events,” Resnick said.

Playtika is one customer taking advantage of the BigPanda solution. Playtika is a digital gaming company that develops social casino games. According to a recent case study, “With BigPanda, Playtika was able to centralize all of its fragmented monitoring tools into a single unified platform that provides instant insight into the health of their IT systems. Moreover, by automatically correlating alerts from multiple systems into high-level incidents, BigPanda enabled Playtika’s IT pros to cut through the noise and quickly detect problematic issues. And as Playtika has acquired companies in recent years, BigPanda has allowed them to consolidate the operations management of these newly acquired companies virtually overnight, with zero migration cost.”

“We were struggling to work with many different tools in order to monitor the performance of our operations,” said Ella Vidra, director of IT Operations for Playtika. “Investigating alerts across our stack was tedious and time-consuming. We were at risk of missing critical errors that could impact production. BigPanda enables us to centralize our IT monitoring and quickly spot the issues that matter most.”

The proof is in the proverbial pudding, of course. There have been plenty of solutions that promise to streamline or simplify logging and alerting. It seems as though we now have solutions that are designed to streamline and simplify the results from the solutions that streamline and simplify other logs and alerts. Ultimately, what IT departments need is something that brings attention to major or critical issues so they can be prioritized without ignoring smaller or less crucial alerts at the same time. BigPanda thinks it can be that solution that helps companies pull together input from all sources and boil it down to a prioritized, manageable list.

— Tony Bradley

October 17, 2016
How Google’s Acquisition of Apigee Impacts Developers

Google revealed a few weeks ago that it is acquiring Apigee for approximately $625 million. The deal demonstrates both the increasing demand for APIs (application programming interfaces) and Google’s desire to remain on the cutting edge of providing a platform for APIs. The purchase also underscores some important trends for developers.

If nothing else, Google’s purchase of Apigee validates the importance of APIs for online commerce, as well as for businesses and developers in general. I had an opportunity to get some insight and feedback from Steve Davis, CTO of Four51, a provider of B2B ecommerce solutions, regarding the Apigee news.

As DevOps and microservices continue to gain momentum, APIs play a very pivotal role. According to Davis, the API is the product for Four51, which makes it the foundation for the company’s DevOps strategies. “Most microservices are API-based as well, so the role is simply that they are the heart of these strategies. The concepts of DevOps relate to continuous delivery and all of the stages from development to deployment and operation. APIs allow developers the tools and visibility they need to work in this environment of delivery.”

I asked Davis if he believes that it was a smart decision by Google to acquire Apigee at this time. “It was a wise move by Google to jump in, as the API economy is accelerating faster than other areas,” Davis answered. “The corporate customer appetite is increasing for APIs as companies continue their digital transformation, and API management is a critical discipline for security, scalabilty and reliability.”

Davis pointed out that Google’s primary competitors—Microsoft and Amazon—have been out there with AWS and Azure cloud services for a while, and they’re doing quite well. He suggested that Google has some catching up to do, and that Apigee—as a best-of-breed independent platform—made a lot of sense for Google to target.

For developers, Google’s acquisition of Apigee elevates Google as a cloud-hosting option—putting it on a more even footing with Microsoft and Amazon. The presence of an API management services enables developers to offload that tedious work and focus on more important things—such as continuing to innovate and bring value to their products.

Davis also shared some thoughts on what the Apigee acquisition means for Four51. “Though we aren’t directly impacted by this acquisition, it does validate the path we’ve taken with our API-first eCommerce platform,” he said. “Many of the same enterprises looking to tap into the API economy are doing so to improve security, scalability and efficiency in their e-commerce and order management with their customers. We’re a big believer in the benefits of cloud infrastructure, as we recently moved hosting of our OrderCloud.io platform to Azure, and we’re utilizing the services there.”

DevOps and microservices are all about streamlining productivity and operating as efficiently as possible. APIs play a major role in that effort, because they allow much of the heavy lifting on the back end to be automated. The acquisition of Apigee makes Google more competitive with its primary rivals, and highlights just how important APIs are—and will be for the foreseeable future.

— Tony Bradley

September 28, 2016
HashiCorp Vault Enterprise Secures DevOps Infrastructure
HashiCorp, a DevOps infrastructure company known for its Atlas platform that combines a variety of powerful open-source tools, recently announced Vault Enterprise. HashiCorp describes Vault Enterprise as a comprehensive security solution for DevOps infrastructure. Vault Enterprise is the foundation for security in the HashiCorp DevOps infrastructure suite.

Open source is a virtual necessity for effective DevOps—forming a close symbiotic relationship. I explained in the DevOps and Open Source report published earlier this year, “There are some significant advantages with FOSS (free and open-source software). With a proprietary, closed-source application there are very few people with access to look at the code. Recent incidents with SSH backdoors demonstrate that closed most certainly does not mean secure. When the code is open those types of backdoors can be quickly spotted and resolved.”

With the addition of Vault Enterprise, the HashiCorp suite is now comprised of:
- Vault Enterprise – Empowers security teams to collaborate, validate, and enforce security policies
- Terraform Enterprise – Empowers operations teams to collaborate, validate, and provision infrastructure
- Nomad Enterprise – Empowers development teams to collaborate, validation, and deploy applications
- Consul Enterprise – Empowers all teams to monitor the complete stack
Vault Enterprise is not entirely new. It builds on the open-source Vault to enable security teams to define and enforce security policy and secrets management across distributed infrastructure.

“The shift to highly elastic, microservice architectures requires a different approach to security. Static network-based security is poorly equipped to handle the dynamic application-centric infrastructures being built today,” said Armon Dadgar, co-founder and CTO of HashiCorp. “Vault Enterprise enables organizations to adopt DevOps practices in their approach to security and keep pace with development and operations teams that are adopting DevOps.”

Vault Enterprise includes Hardware Security Module (HSM) integration, which enables developers to integrate Vault Enterprise with HSMs using Vault’s simple API rather than more complex HSM interfaces. According to HashiCorp, there are also a variety of enhancements and new collaborative features in Vault Enterprise, including a new UI for managing secrets, health alerts and monitoring and an intuitive workflow for initializing new Vault clusters.

“Deploying enterprise software to both public and private cloud infrastructure requires a security solution that works everywhere. As we adopt DevOps practices that make our applications and infrastructure more dynamic, we need bullet-proof security,” said John Mitchell, Architect at SAP Ariba. “HashiCorp’s Vault is the only offering that meets these requirements—one workflow across providers and dynamic secrets management.”

DevOps is a culture and a process—not a silver bullet solution or a single product. HashiCorp understands this while still trying to provide a DevOps infrastructure platform that’s as complete as possible. HashiCorp pulls a variety of effective and useful leading tools together, and delivers them in a modular fashion that allows you to embrace DevOps at your own pace.

There are plenty of vendors out there creating proprietary tools and platforms. What I like about HashiCorp is how it is harnessing the power of an array of open source tools to provide a whole greater than the sum of its parts. Vault Enterprise is available now.
September 9, 2016
Cobalt Merges Bug Bounties, Continuous Testing for Better Security

The rapid pace of application development today can be hard to keep up with, especially when it comes to security. Combining the benefits of continuous testing with the incentives of a crowdsourced bug bounty program seems like a potentially effective way to address that volatility effectively.

Companies need to keep up with demand while staying one step ahead of the competition. It’s very easy in such a dynamic environment for security to fall through the cracks, or be ignored entirely. Combining DevOps-style automation with crowdsourced intelligence is a good approach.

Bug bounty programs have matured and gained mainstream acceptance in recent years, thanks in large part to the leading champion of bug bounty programs, Katie Moussouris. After launching Microsoft’s successful bug bounty program, she left to join Hacker One as Chief Policy Officer. She recently ventured out on her own as a bug bounty evangelist and consultant under the banner of Luta Security.

The concept is simple: Rather than pretending vulnerabilities don’t exist, or sitting around waiting for the bad guys to exploit the vulnerabilities first, a bug bounty program provides financial incentives for hackers and/or security researchers to identify and report flaws. The net result is a win-win-win that results in more secure applications, happier customers and satisfied security researchers who feel appreciated—and paid—for the work they do.

That is where Cobalt comes in. “I’m impressed by how Cobalt has built its model by taking the best elements from the bug bounty space, which offers rewards to those in the security community who can identify software vulnerabilities, and combining them with a scalable, continuous penetration testing platform,” said Robert Fly, an investor and advisor with more than 15 years of application security experience from Microsoft and most recently Salesforce, where he built the product security team and was VP of Security Engineering.

“Companies building web and mobile applications need the flexibility to quickly develop new applications that delight their customers, while ensuring that their applications are secure,” said Jacob Hansen, CEO of Cobalt. “Instead of needing to invest in security tools, such as scanners and penetration tools that are expensive and vary in quality, and having to hire security staff or consultants, our customers rely on Cobalt to provide transparent continuous application testing services, leveraging a powerful platform along with a global community of security researchers.”

There are a variety of well-known software-as-a-service (SaaS) companies already using Cobalt, including GoDaddy, Wix, Weebly, Nexmo and Optimizely.

At face value, this seems like an awesome strategy. Only time will tell how it really plays out, but the success of bug bounty programs when it comes to finding and fixing vulnerabilities in general is indisputable, and continuous testing is a staple of DevOps environments. Combining the two should raise the bar for application security in general.

August 19, 2016
Simplify and Streamline Hybrid Cloud with DevOps

The cloud is not new. The virtues of the cloud have been proclaimed incessantly for years, and most businesses have embraced cloud servers and services on some level. The realities of data security and the need to leverage existing investments in IT infrastructure, however, generally result in some sort of hybrid cloud setup, with servers and applications that span both local data centers and cloud providers. Hybrid cloud deployments can be complex and volatile, which is where DevOps comes in.

I wrote a report recently for devopsy.kinsta.cloud that takes a closer look at the pros and cons of hybrid cloud, and the symbiotic relationship with DevOps. “DevOps Enabling the Hybrid Cloud” examines how DevOps tools and practices streamline management of such an environment. It also looks at some of the security implications and how DevOps can address those concerns.

The report explains, “Speed and flexibility are imperative for organizations that want to remain competitive. Organizations that strive to lead understand that IT must be viewed as a strategic advantage—a powerful tool for driving innovation and creativity. The hybrid cloud provides a bridge from legacy hardware and assets to cutting-edge concepts such as containers and serverless computing for organizations that do it right.”

One truism addressed in the paper is that customers—whether they’re external customers of your company or the users within the company that rely on IT services and applications—don’t know or care where your applications or data are delivered from. They know that they want stuff to just work when they need it to work. It’s up to IT to determine the most efficient and effective way to achieve that goal, and to provide a seamless, frustration-free experience. Sometimes that means relying on cloud servers and services, and sometimes it means using local infrastructure.

I spoke with Justin Foster from Trend Micro about the overlap of hybrid cloud and DevOps, and specifically about unique security issues with hybrid cloud environments. He stressed that legacy tools and traditional approaches to security are not sufficient for the more dynamic hybrid cloud environment. As explained in the report, “Security has to be baked in from the beginning and throughout the development life cycle.”

A pure cloud infrastructure is unlikely for any established company. It may be achieved eventually through attrition, but there’s no getting around the fact that there already are servers and applications in place. The hybrid cloud approach is the right solution in most cases, and DevOps tools and practices are essential to help manage and secure the hybrid cloud effectively.

Click here to download the report: “DevOps Enabling the Hybrid Cloud.”

August 17, 2016
Citizen Developers Can Benefit an Organization

In many organizations, IT resources are pushed to the breaking point while users increasingly demand new technologies, features and capabilities. Users who know a thing or two—often just enough to be dangerous—sometimes “expedite” matters by finding their own shadow IT solutions. Organizations can harness those efforts, though, as citizen developers by finding a way for IT and rogue users to work cooperatively toward a common goal.

Shadow IT can be a serious issue. Consequences arise when users store sensitive company data in the cloud using unsanctioned consumer-oriented services such as Dropbox or track customers or sales in a Google Sheets document. The company loses control and oversight of its own information because it no longer knows where it is, and IT can’t adequately protect data or processes it isn’t even aware of. Users who implement maverick solutions put the company and its data at risk.

On the flip side, these users understand best what is needed to get the job done. They’re the ones working in the trenches and whose employment depends on getting their job done effectively. IT might select or develop tools and mandate that users adopt them, but if they don’t deliver all of the capabilities users need—or worse, get in the way of efficient productivity—users often will take things into their own hands.

What if IT and employees work together? That is the promise of the citizen developers movement. It’s a recognition that IT has neither the resources nor the vision to deliver everything users need, while providing rogue users with a sanctioned framework for creating solutions that IT can approve and manage.

Michael Beckley, CTO of Appian, explained that “shadow IT” is a pejorative term used to describe business teams bypassing IT and enterprise architecture governance to purchase systems that may create disconnected silos. Citizen developers can leverage shadow IT or sanctioned IT. Most successful IT organizations embrace low-code platforms and other technologies that enable citizen developers to operate securely and transparently within IT’s domain.

“The best-designed products have always been built with the needs of the user from the outset,” Beckley said. “Who better than citizen developers to design software that enables organizations to transform customer experiences, launch compelling new products and manage risk efficiently? Citizen developers are motivated by the opportunity to create a new reality, not merely accepting the workplace as they found it.”

Organizations typically enable citizen developers by providing a cloud-based, low-code platform that enables IT to create managed integrations to core systems, and maintain governance and security without significantly limiting the creativity of users.

“Citizen developers are most successful when supported by IT,” Beckley said. “One critical area where IT can help is in promoting reuse and modularity in the applications and business processes that citizen developers build.”

Having worked in the trenches as an IT admin, I have firsthand experience dealing with maverick users who try to circumvent policies and go around IT. At the time, I saw that as an attack on my authority in my role as an IT admin. A lot has changed since then, though—both with the technologies and services available, and my own attitude. I have come to recognize that users know best what they want and need, and there should not be an adversarial relationship between IT and users.

By embracing citizen developers, organizations can harness that ingenuity and innovation to help make the whole company more productive.

August 4, 2016
Black Duck, HPE Partner to Protect Open Source
Open-source software has a number of significant benefits. For starters, it’s both free, which is hard to beat, and open, which means developers can customize or modify it to fit their needs. One issue with open-source software, however, is security. Black Duck has established itself as a leading tool for managing security of open-source tools—and now that protection is extended to HPE Security Fortify.

“Use of open source has increased dramatically in the last five years because it cuts development costs and accelerates time to market. Open source is ubiquitous worldwide and can comprise 50 percent or more of a large organization’s code base,” noted Black Duck CEO Lou Shipley in a press release announcing the HPE integration. “By integrating Black Duck Hub with HPE Security Fortify, customers will have visibility into and control of the open source they are using and also be able to identify known vulnerabilities. This allows them to better understand and reduce their security risks.”

Black Duck lists a variety of key features and benefits of the HPE Security Fortify integration:
- Deep Discovery of Open Source: Rapid scanning and identification of open-source libraries, versions, license and community activity powered by the Black Duck KnowledgeBase, a comprehensive open-source database containing information on more than 1.5 million open-source projects and more than 76,000 known open-source vulnerabilities.
- Comprehensive Identification of Open Source Risks: Create an inventory of all open source in use and a map to known security vulnerabilities, identifying and prioritizing the severity of the vulnerability and exploring remediation steps.
- Integrated Remediation Orchestration and Policy Enforcement: Open-source vulnerability remediation prioritization, mitigation guidance and automated policy management, allowing organizations to have visibility into their remediation efforts and manage their external and internal compliance mandates.
- Continuous Monitoring for New Security Vulnerabilities: Ongoing monitoring and alerting on newly reported open-source security vulnerabilities.
The problem with open-source software security isn’t the software itself—at least not in my opinion. It’s ownership and responsibility. With proprietary software there is no question of who is responsible for addressing any vulnerabilities and developing the necessary patches. But with an open-source project, where hundreds or thousands of developers are contributing to a single platform or application, nobody is truly responsible and, yet, everybody is.

Don’t get me wrong. In most cases and with most open-source applications flaws are addressed and fixes are developed and deployed very quickly. With some open-source code, though, that is not the case. Ultimately, the burden for ensuring that open-source applications are secure falls on the IT managers at the companies using the software.

An even bigger issue than open-source applications themselves is the use of code modules or snippets within other applications. Again, there is a huge benefit to being able to “crowdsource” code and leverage what you need from an open-source community, but if a critical vulnerability is later discovered and fixed in the originating code you also have to take responsibility for updating it in your custom code that uses it.

Jason Schmitt, vice president and general manager for HPE Security Fortify at Hewlett Packard Enterprise, said, “This integration with Black Duck complements our existing secure development and security testing solutions by providing the ability to view the results of open-source scanning alongside application security testing results to deliver a more complete and effective approach to managing application security.”

Open-source software is just about everywhere, which is why IBM, Microsoft, Red Hat, Docker and now HPE all have embraced the Black Duck Hub. Black Duck may not be the only option on the table, but it is the open-source security option that seems to have the most traction right now.
July 28, 2016
AppOps: The Next Big Thing or Just the Next Buzzword?
I’m not sure if you’ve noticed, but the pace of the world seems to continue accelerating. Organizations—and the developers and IT teams that work in them—need to be able to adapt and respond quickly to changing demands. DevOps addresses this challenge for app development and IT infrastructure. However, a new application lifecycle equivalent is emerging, called AppOps, which seeks to bring the same sort of efficiency and agility of DevOps to existing IT services.

Just as DevOps creates an automated life cycle between software development and IT operations, AppOps streamlines application analysis, deployment and management. Sparked by the emergence of BYOD and the consumerization of IT, AppOps simplifies how IT securely delivers and manages existing enterprise and browser-based apps, while quickly adapting to demands of line-of-business managers.

“For a long time now, businesses have shifted toward ‘commercial off-the-shelf’ software packages in hopes of saving resources,” says Calvin Hsu, vice president of Product Marketing for Windows App Delivery at Citrix. “However, this shift toward packaged software doesn’t change the operational needs for IT. They need a fast, simple way to securely roll out and deliver new applications in alignment with operations—an AppOps approach.”

According to Hsu, AppOps provides a variety of potential benefits, including:
- IT can more quickly respond to demands from line-of-business managers.
- cloud-based management infrastructure can play a more significant role in accelerating IT deployments.
- A virtualized secure browser can make browser-based and SaaS apps more secure, and even lock down browser vulnerabilities on apps purchased outside of the IT process.
- Improve communications between IT organizations and executive teams to improve planning and alignment between business and IT objectives.
- Improve service quality by automating the application analysis/deployment process to initiate a constant feedback loop (metrics, user experience, stability, etc.), that keeps everyone related to the app life cycle up to date and enables truly proactive management.
Hsu says that app and desktop delivery simplification and collaboration between vendors is simplifying how IT deploys and delivers apps to employees—virtually, centrally, securely and reliably. He adds that cloud-based management infrastructure makes it easier for IT to offer a menu of services to help business managers determine which apps and data resources will work best for their teams.

On the one hand, AppOps seems like a natural evolution of the DevOps culture. On the other hand, it also seems like it’s just an extension of DevOps rather than its own separate thing. It is simply taking the DevOps culture—breaking down silos and removing roadblocks to streamline collaboration—and applying it to existing IT services.

What do you think? Is AppOps the “next big thing” or just a new buzzword being applied to the same old DevOps?
July 7, 2016
Should Citizen Developers be Welcome in DevOps?

“We’re all developers now. Citizen developer activities will accelerate.” That’s what Gartner predicted back in 2012. Fast forward more than four years and we’ve seen the DevOps movement explode, while the rise of citizen developers has also marched steadily. The challenge for organizations is to figure out how these two philosophies co-exist and complement one another.

What are “citizen developers”? The 2015 State of Citizen Development report defines them as, “… empowered problem-solvers from the various lines of business who have the drive and determination to engage in app development even though they lack traditional coding skills.”

DevOps is about agility and momentum. Automation. Continuous continuousness. The culture and tools that comprise DevOps are all about moving faster—facilitating better collaboration and streamlining processes to accelerate the development and deployment of applications. That more rapid pace relies heavily on automation, in most cases, and still has rules and procedures to manage how things are developed, tested, integrated and deployed.

The citizen developer movement, on the other hand, is sort of an evolution of rogue or shadow IT. There has always been “that guy” in any given company. The one who is too impatient to follow procedures or wait for IT to deploy a solution, and who knows just enough to be dangerous and brew his own unsanctioned tools. Citizen developers harness that same “maverick” attitude, but in a more acceptable and manageable way.

Platforms like Intuit Quickbase or Genesis from Metavine provide users with little or no coding experience with the tools to create effective applications without writing a single line of code. It may seem like that encroaches on the work of developers and confuses things, but there are two caveats to that. First, the users are in the trenches and have an intimate understanding of the issues they’re trying to solve—so theoretically they’re best-suited for solving them. Second, if citizen developers can use code their own solutions for the “low-hanging fruit,” then developers are free to focus on solving more complex business problems and developing more innovative solutions.

Metavine explains, “This approach for generating Cloud and IoT applications places more responsibility in the hands of the people who define business needs, freeing them from typical engineering constraints. Application development becomes more collaborative and highly creative, involving technologists, solutions architects, designers, and line of business people performing cross-functional work to produce agile business results.”

Craig Sproule, founder and CEO of Metavine, provided a description of Genesis. “It is a true zero code application development platform designed to enable the creation of enterprise-class applications. It allows developers and architects to build applications to meet exacting needs, while delivering unprecedented agility to the business.”

While at first glance it would seem that having a bunch of rogue citizen developers implementing their own home-grown apps is a bad idea and would complicate and conflict efforts to streamline development through DevOps. But the reality is that they can co-exist peacefully. Using the right tools and platforms, citizen developers can help solve some of their own problems more quickly and let developers focus on solving more complex problems.

July 6, 2016
Give Test Automation the Respect It Deserves
One of the key mantras of DevOps is “continuous.” Effective DevOps relies on automating routine tasks. Any step in the process that requires human intervention or manual input becomes a roadblock that can slow or halt productivity. To the extent that it’s possible, anything and everything should be automated—and that includes testing.

Test is like the Rodney Dangerfield of DevOps—it gets no respect. It should, though. Test—and test automation—is a crucial element of a successful DevOps program. A recent Gartner survey found that 50 percent of respondents listed test automation as one of the top enablers of DevOps success.

Joan Wrabetz, CTO of Quali, is focused on automating test capabilities and ensuring that testing gets the respect it deserves in DevOps. DevOps isn’t really a process with a beginning and end. It’s a continuum—a self-perpetuating cycle. When test teams take the lead, they need to find ways to push automation back into development and forward into staging and production. “In fact, the problem of test automation is particularly difficult, and this means that the team that implements test automation is in a great position to lead DevOps initiatives beyond test,” Wrabetz says.

That self-perpetuating cycle “starts” with planning and development and “ends” with deployment into operations, but that flows right into monitoring and management, which leads to discovering issues and identifying new feature requests—and that takes you right back to planning and development. If done properly, test falls somewhere in the middle of that cycle, so it’s imperative to streamline and automate that functionality as well.

“Sometimes test drives DevOps. Test organizations have been focused on automation already for 10 years, so often they are the driver for DevOps,” explains Wrabetz. “When test is the focus of DevOps, it is called continuous integration. However, it is also important to work to continuously expand automation outside of test.”

Key Drivers

According to Quali, there are five key components that are essential to successful automation in a DevOps environment:
1. The infrastructure. It starts with one very important characteristic that is critical to testing and often is not automated—the ability to automatically mimic the production infrastructure in which the software (or hardware) being tested is expected to run. Having the ability to give each tester (or set of automated tests) their own personal replica of the target production infrastructure—a.k.a. a “sandbox”—is critical to ensuring that the software coming out of test can be deployed automatically and continuously into production. Automating tests is not nearly as difficult as automating the underlying infrastructure in which those tests run.
2. The production environment workload. Often not considered in automating testing is the requirement to mimic the workloads that would be common in production. This includes the network traffic, other application workloads and network security profiles. These activities need to be automated within the sandbox to create a more realistic environment for testing.
3. The Tests themselves. Of course, the tests themselves also need to be automated and there are a variety of tools to assist in this process. As software gets more complex, this continues to be a challenge, with mobile testing replacing GUI testing as the most challenging type of testing to automate.
4. Reporting and result analysis. To provide continuous automation from development through testing to delivery, it is important to see and analyze test results automatically.
5. Tool Integration. Automation into and out of testing needs to be enabled. This means sandboxes and test automation suites need to be API-driven so that they can be started by a previous tool in the DevOps tool chain and can also initiate a tool that follows testing in the tool chain.
Show Some Respect

Testing is a crucial part of the DevOps continuum. It needs to be done properly and effectively, but it also needs to be a streamlined process that drives DevOps forward rather than a hurdle that slows everything down.
July 1, 2016