Do a quick search on “people weakest link” and you’ll see a raft of articles in which cybersecurity experts and computer scientists point to employees or end users as the biggest vulnerability. No doubt, people do silly (or outright abusive) stuff and open the door for a variety of enterprise security problems. Spoiler alert: There’s this technology called the Internet of Things (IoT) and it’s a bigger security risk than people ever were. The new theme of this rapidly changing IoT security story is data integrity.
Increased cybersecurity risk is due in part to the variety of environments IoT applications touch. Most at-scale IoT deployments use some form of a three-tier architecture, with edge, gateway and cloud components.
Edge devices are those connected sensors and actuators touching the real world. They are usually small, low-power devices often running a wireless protocol such as Bluetooth, Thread, ZigBee, Z-Wave, sub-GHz, low power wide area (LPWA) networks, cellular IoT technology, Wi-Fi or other protocol.
Gateway devices are hubs that groups of sensors connect with. One big role of the gateway is to translate the various protocols from edge devices into Internet Protocol (IP) for transmission into an enterprise network. Gateways also perform a key role in provisioning edge devices securely. Smartphones running Android or iOS often serve as a gateway for consumer IoT devices. For industrial IoT devices, multiprotocol gateways typically are built on a small form factor computer running a real-time operating system or Linux. Edge computing or fog computing architectures use enhanced gateways for more algorithm processing.
Cloud platforms are very familiar to most enterprise developers. In IoT apps, data from the edge arrives for processing, storage and presentation. Algorithms can run on a server cluster or in a distributed computing framework such as Hadoop or Apache Spark. Business intelligence and predictive and prescriptive analytics also reside in the cloud.
In our practice, we see many IoT cybersecurity issues that can be mitigated with proper attention to two areas. The first is shutting down simplistic endpoint attacks by using basic connected device security measures. Avoid edge devices that ship with a well-known default password or PIN—compromising those defaults can be as easy as a battery pull and a factory reset. If a device uses encryption, make sure it doesn’t use a hardcoded key that, if cracked, compromises an entire family. Look for edge devices using over-the-air (OTA) programming allowing for security updates over a long life cycle. As more device developers gain IoT security experience and provide the right features, closing these basic weakest links is easy.
Now, for the more startling weakest link. When the IoT application runs, is the data any good? According to a just-released survey of 950 IT and business decision-makers, only about 48 percent of companies are confident they can detect an IoT security breach. (BTW, don’t get me started on blockchain; it’s one tool, not a panacea.) A lot can happen before your people get involved. By then, it may be too late.
Data integrity needs constant monitoring during at-scale IoT deployment, not just during pilot. An easy way into an unprotected IoT network is to add an unauthorized edge device at a gateway, then use it to access other devices. Gaps in end-to-end encryption are also easy exploits. Untrustworthy public cloud elements are also bad news—a big reason we recommend hybrid cloud architecture. A private cloud layer gives you more control over algorithm performance and data stream security. Presentation and archiving can be left to a public cloud layer.
Bad data can come from either a compromised or faulty device. It propagates quickly across a system, and when an unwitting user at the presentation layer runs a business intelligence tool on it, the bad data could cause issues. How do teams hunt down vulnerabilities and faults that lead to corrupted data and results?
Hire a penetration tester, preferably a contractor removed from your day-to-day development who can objectively assess your architecture during pilot and reassess it during deployment.
Ensure your architecture can tolerate edge device or gateway failures without ripple effects in algorithms; in other words, protect the application from prolonged operation on bad data streams.
Before turning AI loose on automated decision-making, consider an AI-based IoT security analytics tool to spot anomalies and quickly bring people into the loop.
Retarget at least part of your cybersecurity experts to ongoing data integrity efforts and help them stay abreast of IoT technology and trends.
This all sounds like a good job for DevOps, doesn’t it? You know what to do about people; now you’ll need to work magic with devices. The long-held security axiom that people are the weakest link is no longer true when IoT applications run unattended all the way to presentation. There are a few exceptions when people have more interaction at the edge, particularly in the medical device arena. Generally, we’ll see bigger and bigger IoT applications with more data, more processing and less interaction with people.
If someone on the IoT says there’s a security hole you can drive a truck through, they may mean it literally. Data integrity measures will be critical for DevOps teams moving forward. Do you agree? What types of data integrity issues are you seeing in IoT applications?
The testing landscape in DevOps is a big one, with lots of established players feeding the machine. SmartBear is one of those companies. The force behind Swagger, SoapUI and others is perhaps not as well-known as its brethren in the DevOps testing space, but it cuts a wide swath. And, with its recent acquisition of Zephyr, it’s about to get wider.
In this DevOps Chat, we speak via video with Ryan Lloyd, VP of Products at SmartBear, about the Zephyr acquisition and the DevOps testing and test management space in general. As usual, the streaming video is immediately below, followed by the transcript of our conversation.
Transcript
Alan Shimel: Hey, everyone, it’s Alan Shimel, DevOps.com, Container Journal, Security Boulevard, and we’re here for a great interview, hopefully, with Ryan Lloyd, VP of products for testing and development at SmartBear. Ryan, welcome.
Ryan Lloyd: Thank you. Great to be here.
Shimel: Great to have you here. So, Ryan, before we jump into today’s news that we wanna discuss, I wanted to just quickly kinda do a level-set. SmartBear is a company I think most people have probably heard of it, but they may not be 100 percent sure of the full breadth of the SmartBear portfolio. So why don’t we talk a little bit about who is SmartBear and what are the kind of products and services you guys offer?
Lloyd: Yeah, absolutely. So SmartBear software is a company with a lot of history and if you don’t recognize the company brand, it’s very likely that you recognize one or more of our product brands, so we are the company behind some of the key product brands like Swagger, SoapUI, CrossBrowserTesting, TestComplete, so quite a diverse portfolio of products focused on helping our customers who’re building software. Help them build tests and monitor their software applications.
Shimel: Great. So, Ryan, big news out today is SmartBear had made another acquisition. You guys have acquired Zephyr. Is that correct?
Lloyd: That’s right. Yeah, we’ve announced our intent to acquire Zephyr, a company in a similar market space as us.
Shimel: So it’s a similar market space, but let’s get specific. What does Zephyr do?
Lloyd: Great. So Zephyr is a collection of tools for testing and specifically focused on the area of test management, managing the process of testing.
Shimel: Great. And you mentioned an intent to inquire – acquire, so I guess that means it hasn’t yet closed, but you’re both private companies and I would imagine it closes fairly quickly.
Lloyd: That’s right. We expect in the next 30 days, subject to the usual regulatory approvals.
Shimel: Yep. Fantastic. So, you know, Ryan, one of the things I read in the background around this one was that one of the big pluses here is Zephyr’s presence within the Atlassian, the Atlassian ecosystem, and SmartBear actually is already a big player in that Atlassian ecosystem. Well, aren’t they?
Lloyd: Yeah, so we have a couple of products that have plug-ins to the Atlassian ecosystem. I mean, the reality is 60, 70 percent of our customers are using Atlassian’s tools. They’re pretty dominant in the area of software teams, for managing some part of the software development process, so it’s a very strategic and important ecosystem for SmartBear as a company. And, certainly, Zephyr improves our ability to reach that ecosystem of users and make sure that they’re aware of all of the tools and capabilities that SmartBear can offer.
Shimel: Yep. And, Ryan, you know, 60 to 70 percent use Atlassian. I’m gonna imagine the overwhelming majority are using Jira and then maybe Jira Plus.
Lloyd: Jira –
Shimel: Correct?
Lloyd: Certainly, Jira is probably the most prevalent. Confluence gets tagged on with Jira quite a lot. And then BitBucket, actually, is also becoming quite popular as well. BitBucket, GitHub, GitLab – they’re sort of three of the same, in terms of enterprise solutions for developers.
Shimel: Got it. So I don’t wanna turn this into being about Atlassian and their products, but, obviously, they do have a big ecosystem. There are a lot of third parties, such as SmartBear and Zephyr, who move a lot of services and product via that marketplace. But let’s talk a little bit similar, but where’s the overlap? Or is there any overlap between what Zephyr and SmartBear’s offerings are about?
Lloyd: Yeah, so SmartBear offers a broad portfolio of products and, certainly, a couple of the products we have, pre-acquisition, were in this test-management space. What Zephyr brings to the table, though, is a solution that is natively built around Jira and delivered through the marketplace. So what that means, to be delivered through the marketplace, is it’s a native Jira solution that Jira administrators and customers can install directly from within their Jira instance. So it provides a very seamless extension of their workflows, within Jira, to handle some of these test-management scenarios, which we think is really important to teams that are trying to consolidate their processes and have a unified approach to that.
Shimel: Excellent. I guess one of the issues – and, with your title and your responsibilities around VP of products for test-and-dev, just in a number, how many products are there in test-and-dev now, within SmartBear and then if we add the Zephyr suite to the mix?
Lloyd: Yeah, so we had five products in that portfolio. Today, Zephyr brings three additional products into that portfolio, so we’re now up to eight products in our test and development business, as well as a range of products in our other business unit, around API.
Shimel: You know what? That’s a lot of products to be managing, huh?
Lloyd: It is. I mean, SmartBear has grown a lot over the years, through both acquisition of companies and building new organic products and bringing them to market. It’s pretty exciting time for us. We’re, pre-acquisition, at about 450 employees globally, so it’s an exciting time to be part of this company.
Shimel: Absolutely is, man. So, you know, another not necessarily unique but another kind of factor you have to play into the SmartBear – and I don’t know about Zephyr as much – is that some of your solutions have open source kind of versions out there that you guys help manage the open source communities around these, as well as the commercial versions of the products themselves. Correct?
Lloyd: Yeah, that’s right. I mean, core to our business is really a couple of philosophies. One is we wanna be open and interoperable with all the tools that are part of the ecosystems that software teams are using. And so, in some cases, that means commercial opportunities to partner with companies like Atlassian; in other cases, it means we want to be a part of an existing open-source ecosystem or help build up and nurture an open-source ecosystem. So, certainly, when we brought SoapUI and Swagger into our product portfolios, that was all about really taking that existing organic ecosystem, where developers and testers are going through tools, and helping build and nurture and develop that ecosystem.
Shimel: Excellent. So what kind of special challenges – you specifically, right? You own the products; you’re VP of products here. What challenges does that pose for you, having these open source kinda versions and tools and communities. I mean, you don’t wanna burn them, right? Because, ultimately, you kinda bite the hand that feeds you. So let’s talk about that. What does that – how does that manifest itself with you?
Lloyd: Yeah, certainly. I think there’s a brand and a culture that exists around open-source products and I guess it’s our responsibility, as stewards of those brands, to really continue to nurture that community. And that means we’ve gotta make balanced decisions about where we choose to add features to the open source products, versus where we commercialize some of those features, so there’s a constant balance there and kind of evaluating and making sure we’re engaging with the community.
And then, more broadly, beyond just our open-source products, I think one of the biggest challenges in the software tools space and market is the interoperability that’s required between tools these days. You know, developers and testers, they’re demanding practitioners. They want their tools to work together and there’s a huge diversity of tools out there in the market. So, as a product manager, that’s what keeps me up at night, is making sure all these tools are compatible and working to support these complex workflows.
Shimel: Absolutely. I mean, one way of looking at it is, if there wasn’t such a big market, there wouldn’t be all these tools.
Lloyd: That’s true.
Shimel: Another way of looking at it is also that it presents a tremendous opportunity for consolidation and for one or a few products to emerge as true market leaders. And then you’ll see it kinda – you know, you’ll see the amount of products in the market kinda consolidate. You know the old saying, “If you’re not – you’re either leading, following, or get out of the way,” right? If you’re not top-three, you gotta ask yourself, “Do you wanna stay in that market?” But, you know, when we talk about the testing market – and, in DevOps, we talk about continuous testing – it certainly is a huge, huge market here.
Lloyd: Yes, it is. And it’s one that’s continually evolving and shaping, right? So there’s traditional test tools that have been around for a decade or more, and then there’s sort of emerging products that are coming to market. And I think it’s how all these work together and evolve to support the change in workflows and, yeah, even just organizational changes within environments.
Shimel: Yep. So another area of the testing space, Ryan, that I’m interested in personally is security and security testing. And it’s really funny when I – you know, 20 years ago, when I was into security, when I first got into security, info-sec, QA was here and security was there and never the twain shall meet, right? But now we’re starting to see security become synonymous with quality – that’s the “Q” in QA.
Lloyd: Yeah.
Shimel: And so security testing and other kinds of testings are starting to merge. What do you guys see with that at SmartBear?
Lloyd: Yeah, I think what’s driving a lot of that is the Agile workflows that teams are adopting, so, as they increase the pace of trying to deliver and release software, they’re being forced to evaluate quality and security earlier in the life cycle and more from the context of the development team, rather than having a separate, isolated team that deals with that downstream or after the fact.
Shimel: Correct.
Lloyd: So I think that’s the driver, so definitely a lot of demand for your Agile workflows to support testing by design and security by design. From our perspective, I think that opens up some unique opportunities for adjacencies and areas that SmartBear can continue to expand to in the future.
Shimel: Yeah. I mean, so, you know, what you’re describing is almost the classic “shift left.” Right? As we start shifting testing, security and security testing, left early into the development life cycle. But it’s also the reality of it’s been called the “modern software factory.” Right? It’s an assembly line and is a production line, for software, and where traditional QA, traditional testing, security testing, where they fit into that life cycle has shifted left.
But also is – you know, where we used to think, oh, QA was going away and security people were gonna give up control of security. Not the case. They’ve become more important, right? But also highly automated where they can be and clearly defined as part of this factory, if you will.
Lloyd: Yeah, it’s really just about reshaping the responsibilities and improving the collaboration across the whole unit or team that’s now responsible for delivering iterations of software.
Shimel: So, Ryan, then – so that begs the question then, “What does that mean for you as a product manager?” Right? Where you used to sell to a QA engineer or security testing sold to a security person, but now your audience includes the guys using Jira and BitBucket and a bunch of other tools. It’s not just Atlassian, obviously. So you have new customers to serve and to service. Now how do –
Lloyd: Yeah, that’s absolutely right. Yeah, so, over the last several years, we’ve been focusing on how to modernize our portfolio to support those sort of emerging needs, both the process change that’s going on with our customers and the persona change that’s happening in many instances. So the acquisition of Swagger is a great example of us providing tools that help developers earlier in the life cycle, so, where we had SoapUI to help with API testing from a QA perspective, Swagger aims to solve some of that further upstream by enabling more collaboration around the definition of those APIs to begin with. And, oh, by the way, making sure that interoperates with SoapUI, so you get leverage out of that API definition to generate some of those tests, automate it, and it triggers sort of a seamless workflow and thread.
And so we’re continuing down this path of modernizing our portfolio. We acquired CrossBrowserTesting about two years ago, to help companies address some of the modern types of applications that they’re looking to test, integrated more deeply with Selenium, which has become hugely popular in sort of the developer organizations, to drive automated Web and mobile testing. And then, with Hiptest and Zephyr, again, it’s furthering our relevance for those software teams that are on that journey.
Shimel: Excellent. Ryan, last kind of issue I – or not issue – it’s not an issue at all – but the last area I wanted to talk to you about was so you’ve got five. There’s three more coming in. And that’s just test-and-dev; you have an AP kind of silo as well within SmartBear. How do you – I mean, I’ve been in the boat not quite as many as you, but, when you have so many different products and then a company name, how do you brand? Right? Do you continue branding the individual products? Do you throw SmartBear in front of all of them, but some of ’em already have names that may be, in fact, bigger than SmartBear, as you said in the beginning? Again, as a product manager, you have to think about that a bit. You know, what’s the right answer there, if there is –
Lloyd: Yeah, so I think SmartBear is going through a bit of a transformation, in that regard. You know, as I said and as you pointed out, we’ve had very strong product brands up to this point. We wanna continue to nurture those product brands, so no intent for those to go away because we love the high-velocity, try-buy model we have around our products, where developers and the practitioners can find the right tool for the job and get started.
But we’ve gone through an exercise on how we can integrate our products in the right way, so like the Swagger, SoapUI example I provided, to create those sort of interesting workflows. And we’re building up the SmartBear Academy, our SmartBear community, our big user conference, to help elevate the SmartBear brand and create more awareness of the portfolio of products we have – how they work together, how they fit into the ecosystem. And I think we’re just at the start of that journey right now. We’ll see a lot more, as we launch into the year next year, around what we’re doing from a brand perspective to galvanize the company and all of the great products we have.
Shimel: Excellent. Well, Ryan, we’re about out of time, but, first of all, congratulations on the Zephyr announcement and anticipated acquisition here. It’ll make for some good times. And continued success with SmartBear and everything you guys have going on.
Lloyd: I appreciate it. All right. Great talking to you, Alan.
Shimel: All right. Ryan Lloyd, VP products, test and development, for SmartBear, on SmartBear’s acquisition of Zephyr, here on DevOps.com. And we’ll talk to you soon, everyone. Have a great day.
For DevOps engineers, “noise” is the enemy of productivity. In this context, the noise we’re talking about is unnecessary or low-priority alerts and notifications that distract engineers from identifying serious issues—and ultimately can cause alert fatigue syndrome, in which alerting systems are ignored altogether.
Without the application of a well-constructed noise reduction plan, alert noise will end up taking away from the focus of solving the actual problem that may be causing a more critical issue.
So, how can you fight alert noise? You might be tempted to assume that the answer lies in reducing the number of alerting tools you use, but that’s probably unrealistic. Teams today are dealing with complex application stacks and multiple tools and data sources, all of which can produce their own alerts for issues that may be interdependent or isolated. You simply can’t take tools out of your stack.
What you can do, however, is analyze and manage alerts more effectively. Dealing with alert noise ultimately boils down to developing an effective problem analysis strategy. Let’s explore how, with a specific focus on how AIOps factors into keeping alerts manageable.
Reduce the Time in Analyzing Alerts
While reducing noise is important, DevOps teams risk losing valuable time solving critical problems if they also have to create and/or add systems to properly analyze alerts and their sources and priority level. Maintaining this system also would be a source of unnecessary labor that may reduce the alert noise, but also reduces the amount of time that DevOps staff can work on the solution to the root cause— which brings up another consideration. When coming up with a plan, the amount of complexity involved in adding the monitoring tools necessary to cover an entire application and/or service can be equally laborious compared to not having a solution at all. At this point, we may be merely shifting the labor emphasis.
This is Where AIOps Comes In
Now, let us envision a way that a team can gain full insight into a root cause without exposure to alert noise. The team doesn’t need to analyze every single alert, which would include:
Duplicate alerts.
Alerts from multiple sources (tools, microservices).
Informational alerts (warnings).
Instead, the team can let AIOps analyze and classify the stream of alerts and identify critical issues. DevOps no longer would need to spend valuable collaboration time analyzing every link in the toolchain. When a team has to analyze every tool, the team could produce false positives, redirect resources toward the wrong solution, spend too much time trying to identify the root cause and create excessive alert noise. Let us envision a way in which the team does not need to spend excessive time in analysis and can experience more effective collaboration in service of solving the actual problem.
AI and machine learning can relieve this manual process through AIOps. Automated collection of data across the application stack, alert correlation, identifying root cause and decision-making can be leveraged by a team to get to the problem immediately without all of the messy work of manual analysis, sorting and identifying root cause. This level of automation allows for needed human interaction through collaboration across teams, which can result in faster resolution, decreased downtime and effective communication of key events to company staff and end users.
Analyzing to Predict Future Problems
Another benefit of AIOps and the power of automated data collection, analysis, issue identification and decision-making is the ability to predict future events. A DevOps team can leverage these insights into effective preventative measures. The ability of AIOps to produce these insights will help a team find ways to automate and/or improve operations. Reducing alert noise no longer needs to be a goal because it is happening through a smooth interaction between the human team and the AI.
There is a growing need for DevOps teams to utilize the power of AIOps. While human teams are not replaceable, redundant and time-consuming tasks are. The ability of AIOps to effectively traverse a complex toolchain with isolated issues to analyze alert importance, correlate alerts, collect and analyze relevant data, identify root cause and produce holistic actionable insights eliminates inefficiencies in problem-solving and reduces alert noise.
Making DevOps less noisy is a goal with its roots in effective use of information. Alerts serve to inform, but thanks to the increasing complexity of applications, alerts become noise. Leveraging the power of AIOps serves to do a lot of the dirty work for a team so that noise can be reduced—but more importantly, it can make operations run smooth and keep customers happy.
In 2018, digital transformation ushered in a radical shift in how enterprises harness customer insights, technology capabilities and rapid experimentation to drive revenue growth, profitability and market leadership. Enterprises spent $1.3 trillion in 2018 on digital transformation technologies such as public cloud platforms, microservices and containers, edge computing, machine learning and artificial intelligence to improve customer experiences, business agility and employee engagement.
Heading into 2019, digitalization will accelerate business model innovation and drive the adoption of these emerging technologies for sustainable competitive advantage. Here’s a closer look at five technology trends that will change the way enterprises approach digital operations management in 2019:
Multi-Cloud Adoption Goes Mainstream
In FY18 Q3, the three leading cloud platforms (Amazon Web Services, Microsoft Azure and Google Cloud Platform) accounted for 65 percent of the market share of public cloud infrastructure and platform services. Enterprises are increasingly relying on multiple cloud providers to avoid lock-in and harness innovative product offerings, global presence and flexible pricing strategies. When it comes to multi-cloud adoption in 2019:
Lift-and-Shift Will Give Way to Rebuilding. In the initial phase of enterprise cloud adoption, CIOs preferred “lift-and-shift” strategies for migrating complex, legacy applications to public cloud services. In 2019, enterprises will rebuild legacy portfolios to truly take advantage of the cloud’s elasticity, security and utility pricing. Rebuilding is not only the right approach to modernize legacy systems, but also a viable solution for reducing technical debt and unleashing enterprise agility.
The Rise Of Managed Private Cloud for On-Premises Requirements. While analyst firm Gartner predicts that 28 percent of enterprise spending will shift to public cloud offerings by 2022, the reality is that the vast majority of enterprise workloads still run inside the corporate firewall. Public cloud providers are taking aggressive measures to conquer the final bastion of data center gravity with private cloud offerings. Managed private cloud infrastructure will become essential for enterprise services that will need data sovereignty, latency or compliance, yet still want the flexibility and agility of cloud. Some recent managed private cloud announcements from the big three cloud platforms show how these providers are taking a pragmatic approach to meet customer demand:
In 2016, Andy Jassy, CEO of Amazon Web Services, declared, “Whether it’s 10 years or 20 years, very few companies will own their own data centers, and those that do will have a smaller footprint than they have now.” During his 2018 re:Invent keynote, Jassy sang a different tune by launching AWS Outposts to bring the power of cloud-native services to enterprise data centers for hybrid interoperability and seamless management.
During the 2018 Cloud Next conference, Google introduced its Google Kubernetes Engine (GKE) On-Premises that delivers an abstraction layer for running container clusters either on the public cloud or in a corporate data center.
In 2017, Microsoft announced the general availability of Azure Stack, which offers the same programmable interface and deployment tooling of Microsoft Azure in a private cloud environment for a consistent hybrid cloud experience.
The Cloud-Native Skills Gap Widens
Software containers have fast emerged as the standard deployment vehicle for agile, reliable, scalable and highly available enterprise services. The transition from virtual machine management to container orchestration and monolithic development patterns to agile microservices will transform the way enterprises build, deploy and optimize cloud-based applications. IT operations teams will need to develop a new set of competencies to manage the health and performance of containerized services. IT practitioners can longer rely on manual scripts to manage large, distributed systems but will need to embrace infrastructure as code to provision, deploy, monitor, alert and auto-scale cloud-native services.
Commodity Monitoring Evolves to Strategic Observability
DevOps teams will use multiple data sources (metrics, logs, and traces) to understand, verify and debug cloud-native systems. Modern observability requires the ability to consolidate and extract relevant insights from observability pipelines using tools such as Prometheus for monitoring, Jaeger for distributed tracing and Fluentd for logging. While IT teams will need new tools to manage autonomous cloud-native services, it is still critical to understand relationships between individual components and the overall system to effectively troubleshoot microservices in production.
AIOps Emerges as the Way to Tame Alert Storms
Artificial intelligence for IT operations (AIOps) is a clear necessity for modern IT infrastructure teams to keep up with the diversity, velocity and volume of IT operational and event data. IT teams can proactively prevent service outages, analyze real-time incidents using contextual knowledge and remediate problems using AIOps tools. Here’s how AIOps will help enterprises run and optimize mission-critical systems:
Improve Response Time For Digital Interactions. AIOps has the power to boost key metrics for incident management including mean-time-to-detection, mean-time-to-response, mean-time-to-restoration and incident volume handled within a service window. The combination of machine learning and data science techniques in AIOps not only delivers faster incident coordination and response, but also reduces the human time spent per alert with advanced analytics and probabilistic root cause analysis.
Eliminate Siloed/Redundant Processes. AIOps offers the ability to consolidate event and incident insights from different IT management tools across on-premises and public cloud environments. A shared AIOps platform offers centralized visibility, faster impact analysis and improved collaboration for a diverse set of stakeholders, including application owners, infrastructure teams and business sponsors.
Business-Aware, Service-Centric Delivery Can Optimize Customer Experiences
To help the business remain competitive, IT teams simply must transition from managing hybrid IT infrastructure assets to prioritizing and supporting business-critical services. Service-centric operations management combines business insights, customer knowledge and relevant metrics to improve digital performance, enhance customer interactions and support business goals. IT teams will need to focus on the business value of IT services while offering holistic infrastructure visibility and control at the business process level.
Clear strategy and focused execution will separate the digital winners from losers in 2019. How enterprises embrace and respond to these key trends will be key to unlocking disruptive innovation, operational transformation and business value.
The concept of a “war room” is time-honored. One of the early examples is the Churchill War Rooms, an underground bunker that allowed Britain’s leaders to plot the allied route to victory during World War II. British Prime Minister Winston Churchill, as plain-speaking and self-assured as ever, said, “This is the room from which I will direct the war.”
Today, the war room concept also serves the world of business—specifically, organizations’ IT departments when troubleshooting a major issue impacting the business applications and their supporting IT infrastructure. The problem? So many parts of the infrastructure, whether hardware or software, can be implicated as the root cause of any problem. As a result, IT war rooms can encompass many stakeholders who are experts in those individual parts. And, while the goal is to collaborate to resolve the problem and minimize the damages from it, IT war rooms often result in a case of too many cooks in the kitchen, all speaking different “languages.”
War rooms are a symptom of a much bigger problem within the IT organization. At the highest level, it results when IT operations teams have insufficient visibility and have limited proactive control of the infrastructure. For one thing, in a siloed organization (which most enterprises are), discussion across silos is rare. Plus, siloed monitoring tools for resolving infrastructure issues lack a common context and have no inherent understanding of the business impact of the applications. By contrast, the hardware, software and data that span silos do “talk.” In a sense, they collaborate. That’s the nature of connected infrastructure. Data, for example, flows throughout the enterprise, altogether oblivious of silos.
The lofty view of war rooms, in the words of Dennis Drogseth, vice president at analyst firm Enterprise Management Associates, is this: “Our war rooms can be either physical or virtual. Highly automated or not. Made up of consistent, well-defined teams, or not. But what makes them war rooms is the need for collaborative decision-making across silos.”
How Low Can IT War Rooms Go?
The reality is that, despite their noble charters, IT war rooms have devolved into blamestorming—rancorous debate and finger-pointing to indict application development, database analysts, IT operations or DevOps for a failure. Drogseth described a bleary view of war rooms as “disastrous assemblages of finger-pointing adults caught up with siloed versions of the truth.” In an even more disillusioning example, one vendor stated that “mean time to innocence” becomes the IT war room’s chief metric”—rather than isolating and resolving the root causes of issues.
With so many factors in play, dollar costs of IT war rooms are difficult to calculate. Taking aim at the problem, Enterprise Management Associates said, “The average War Room comprises 15 people and takes 5-6 hours to respond to and resolve the incident that caused it to convene. 29% of War Rooms take more than 11 hours to respond and resolve.” And those figures are over and above the substantial losses that occur if a server or switch, for example, goes down and war room staffers bicker over who’s to blame. Revenue losses can easily reach into the millions of dollars per hour of outage.
Furthermore, the problems and costs are growing. Infrastructure complexity has ballooned with the growth of hybrid data centers. The hybrid model uses cost-effective, flexible compute and storage via the cloud, while retaining the control and security provided by on-premises infrastructure. However, the reality is that highly virtualized, multi-cloud environments are amazingly complex and problematical when it comes to ensuring application service delivery. Out-of-control infrastructure leads to outages and slowdowns—and finger-pointing.
Remedies
There are, however, several ways to mitigate the downsides of IT war rooms:
Discover technology solutions that provide cross-domain insights. Stated more simply, that means finding tools that can trace infrastructure issues to their source—exactly where the issue originates—even if the symptoms appear elsewhere in the infrastructure. Achieving this level of sophistication effectively eliminates finger-pointing.
Learn from history. Issues repeat themselves; if an issue occurs but is not traced to its source and remedied the first time, it will almost certainly recur. Often, it requires analytics-based tools to make the necessary correlations and remedy an issue.
Create what EMA calls an “informed context.” An informed context can help resolve an issue, moving from triage and diagnostics, to remediation, to validation. Knowing where applications live on the infrastructure at any given time and the business priority or tier of each application is one example of “informed context.”
IT will likely always have its war rooms, and many organizations will claim that they’re necessary. At its best, a war room is a highly motivated, solutions-driven SWAT team. Of course, SWAT teams, like anything to be valued, will forever come at a cost to the enterprise. One way to view the IT war room is that it’s just one more cost center. Maybe the better way to view it is by quantifying its value: If a war room saves minutes in getting a server up and running after an outage—or prevents an outage altogether—the savings could run in the millions of dollars. But too often, the opposite is the case: IT war rooms further contribute to the lost revenue from a failure.
Let’s hope that enterprises remember the original intent of IT war rooms and reorient them around strategy rather than blame. If not, enterprises might want to consider closing the door of the IT war room for good.
Brick-and mortar-retail is rapidly transforming with the digital age. As physical storefronts turn digital, it’s safe to say that traditional retail is quickly becoming a mesh of “brick-and-click.”
So, how are retail giants surviving in this new landscape? Interestingly enough, the remaining brick-and-mortar retail are reinventing themselves by applying helpful DevOps principles.
In this post, we’ll peek into cases of innovation and horror stories from historically brick-and-mortar retail chains and brands. From Target, to Balian Group, Sears and other companies, we’ll discover the importance of DevOps in this new brick-and-click multi-channel retail environment.
Target
In an effort to increase agility, Target engineering teams embrace something they call “the Dojo”—immersive, six-week training sessions with hands-on DevOps and Agile coaches. As Target CIO Mike McNamara recently told the Enterpriser’s project: “The Dojo has been fantastic in getting teams engaged with agile and DevOps, removing the natural resistance and fear of change, and then supporting the team through the changes while maintaining productivity. It’s been a huge success for Target.”
The result of these sessions is increased comfort with methods including containerization and tools that enable continuous integration and continuous deployment. McNamara also noted their grassroots approach; they encourage open source activity and listen to their engineers when they petition the use of new tech.
“Our engineers have fully embraced DevOps,” he said.
Bailian Group
For another case study, take Bailian Group, a large Shanghai retailer that operates 6,000 grocery and department stores. By the time Bailian entered the online marketplace, it suffered from a severe lack of agility, non-native web apps and underutilized server assets.
Qichuan and Bailian realized that to stay relevant, they had to cut long development cycles and embrace “new cloud application development tools and processes that foster a CI/CD and DevOps culture and increase innovation and time-to-market,” he said.
They ended up implementing a private cloud using OpenStack. Now, midway through their cloud transition, they estimate to have 10 million virtual machines by the end of 2020.
Sears
While the previous two examples had positive outcomes, digital transformation has not gone over well for other retailers. Sears declared bankruptcy in October 2018.
In its final years, Sears was in a sticky situation. To the chagrin of former CEO Eddie Lambert, surveys found their customers preferred to shop in stores as opposed to online. Disregarding customer desires, however, Sears attempted a big data play with its “Shop Your Way” program, under which Sears gathered user data to then sell. The program floundered, resulting in consumer irritation and loss of sales as reported by Business Insider.
Writing for InformationWeek, Emily Johnson reminded us that evaluating DevOps using the wrong metrics will ultimately lead to failure, citing customer satisfaction as the No. 1 goal.
In its final years, an abrasive Sears leadership halted any sense of DevOps to flourish. A disengaged Sears CEO Lambert created “a climate of fear,” according to Busines Insider reports. It appears the retailers that stay afloat encourage agility and intrapreneurship—this means encouraging innovation, not fear.
DevOps Aids the New Omnichannel Retail Benchmark
Brick-and-mortar are evolving into omnichannel retailers that bridge physical and virtual spaces. Take IKEA’s augmented reality app, which visualizes items in a house. From Disney, to Startbucks, Nordstroms and elsewhere, physical storefronts embrace digital offshoots of their core model to augment the traditional retail experience.
Surprisingly, physical storefronts aren’t completely dead. They still offer conveniences for urban environments with high walking scores. See the emergence of Amazon physical locations for ironic evidence of their benefit. Other storefronts and pop-up shops have taken on a museum-like quality. The flagship Funko store, for example, creates impressive displays and interactive exhibits, pulling in flocks of weekend shoppers that want more than a dull e-commerce shopping cart.
DevOpsGroup similarly describes a “reverse showrooming” movement; consumers research products online to then purchase in person. This enables brands such as ASOS, which is emphasizing social media and free worldwide deliveries, to stay competitive.
“Retailers have been forced to re-evaluate their business strategies and understand how new technologies will affect their offering,” according to the DevOpsGroup website.
DevOps Turns Bricks to Clicks
Online shopping has sufficiently disrupted retail, and brick-and-mortar must reinvent themselves to stay relevant. An open-minded leadership and a commitment to fast development, dynamic scaling, uncompromised availability and open source technology underpin a successful evolution.
Of course, this increased reliance on technology necessitates a hardened look at how development and operations communicate, and brick-and-mortar retailers must adopt the behaviors of tech companies to retain agility.
This may come in many forms: new user-facing digital experiences, microservices for composability in niche omnichannel environments, containerization to package code that scales well, distributed data stores, heightened user data security or smart incident management techniques, to name a few.
Traditional retailers, then, should dissuade their doubts against agile. DevOps can aid retail by reducing redundancy, cutting costs and streamlining production and distribution.
Feature flags, also known as feature toggles, provide a means for DevOps teams to test and add new capabilities to their applications without having to take the entire application offline. The downside is they generally result in more branches of applications that need to be carefully managed within the context of a DevOps process.
A recent survey of 550 software developers and decision-makers conducted by the market research firm Vanson Bourne on behalf of Atlassian and Rollout.io, a provider of a platform for managing feature flags, suggests that IT organizations are getting more comfortable with managing feature flags.
The survey finds 95 percent of survey respondents said their organizations have implemented, have begun implementing or plan to implement features flags. Those that have implemented feature flags said they add 30 new features on average to each application per year, compared to those that have only begun implementing (21) or plan to implement (18) them. A full 81 percent said use of feature flags will increase over the next 12 months, and 88 percent said modern software development should assume to include feature flagging.
The primary driver for implementing feature flags are a need to reduce risk (46 percent), increase speed of development (46 percent) and increase speed of deployment (45 percent). Eighty-seven percent of respondents said enabling an on/off or “kill” switch for features also reduces the risk of application downtime for customers.
A total of 97 percent of respondents said they are under pressure implement new application features faster, and yet 65 percent said it is difficult to accomplish that goal safely. More than 61 percent of respondents said their organization releases new versions or updates for applications once a week or more. However, 72 percent admitted they have lost customers or users due to application development errors.
Rollout COO Ronit Belson said building out the functions required to manage feature flags doesn’t make much sense when platforms such as Rollout.io are now emerging. The survey finds nearly 90 percent of respondents who have implemented feature flags in-house would consider moving to a third-party supplier for feature flagging management software. A total of 37 percent of respondents said the cost of in-house development is higher and 32 percent said they encountered more complexity than anticipated.
Belson added most organizations not only fail to appreciate the cost of implementing feature flagging on their own, but also the impact feature flagging has on the culture of the IT department.
Thanks to the rise of microservices, feature flagging should become more prevalent as monolithic applications give way to a more granular approach to application development that makes it easier to isolate functionality. In fact, the ability to apply feature flagging might be considered one of the primary benefits of the agility inherent to any microservices architecture, especially it if relies on containers that can be easily ripped and replaced. The challenge organizations will face is implementing the DevOps processes required to embrace feature flagging at scale.
In a rapidly changing world, predictability has never been more powerful. It’s also never been so underrated. While it is easy to think about predictability in terms of repetition or even monotony, predictability in business means being able to correctly estimate how long it will take to complete a project and how risky it is. But maintaining predictability means a few other crucial things: better allocation of resources, less overspend on projects and a deeper understanding across teams on what’s happening with a project. All of this leads to one thing every business ultimately has their eye on: profit.
One of the biggest pain points in business is delivering projects on time and on budget. Your latest project might be innovative and essential to the future success of your firm, but the truth is, there’s every chance it will cost more than you think—despite your best budgeting efforts—and take longer to complete. In fact, a study found 66 percent of projects in the software industry were not completed on time, 33 percent were blowing their budgets, and 17 percent were not returning the value stakeholders expected. When we’re looking at 17 percent of projects having the potential to take the entire company down with them, it’s not something to be taken lightly.
This continues to be a massive problem for the software industry. What is needed is more transparency and higher levels of predictability built into processes, which can eliminate obstacles such as project overspend, delays and unhappy customers. I know this from personal experience. We managed to increase the predictability of our core processes by 102 percent in nine months, and as a result, half of our new contracts have been from referrals made by satisfied customers, that in turn received more accurate quotes and time frames for their projects with us.
To achieve this, you will need to establish KPIs that define predictability and then design tools which consolidate and capitalize on this data. It won’t be easy, but you will have more control than ever before—which is the perfect foundation for innovation. It had always been one of our goals to become more predictable and put the necessary tools, processes and metrics in place when COO Marek Talarczyk made it a priority in January 2017. When you can accurately measure how long something will take, and how risky it is, you can plan costs and revenue, and thus become a valuable partner for other business entities. There will also be fewer problems for key stakeholders, which leads to increased customer happiness and trust. Once your predictability data is consolidated on a single platform, you will also be able to use it to make better hiring decisions in the future, or—as we did—develop a traffic light system which uses a machine learning algorithm to identify which projects are likely to have issues, and then isolates the potential cause. Tools such as this helped us almost double in size in 18 months. Before we made this effort, we were often wasting time and money dealing with project crises.
Most importantly, companies who can help other companies become more predictable will lead the way in software, design and digital transformation. It means they can develop a precise plan for their business and marketing strategy in advance of the product launch—which is not based on a best-case-scenario guess, but reliable data. There will also be fewer awkward, dissatisfying meetings about cost blow-outs; your key stakeholders can prepare for the costs of development and won’t be surprised by the sudden need for extra budgeting.
Just remember, the goal is to make accurate predictions on how your projects will play out—and not judge team members based on the KPIs generated by these new tools. Overcoming this concern was one of the first hurdles we faced, but it’s simply a matter of explaining your true intentions. It’s all about solving problems, while predicting challenges and obstacles.
It isn’t news that DevOps and IT security teams often struggle to align their departments and maintain a coherent balance between keeping a business secure and developing new applications to maintain customer interest. While security processes are a necessity, they can be deemed by DevOps teams to be manual and cumbersome, blocking the agility that makes them so effective in bringing their solutions to market. IT teams conversely feel their counterparts are prepared to sacrifice security in the name of innovation and revenue.
Even if both teams do respect the other’s intentions, any conflict could lead to delays in both of their processes. For example, an IT team may need to make crucial updates to the network security and warn different teams they may experience some downtime during this crucial implementation. However, DevOps have typically been given more leeway in how they operate as they are so important in today’s software-driven world, and may ask for the update to be delayed so they can complete tasks or meet deadlines, leaving the IT team waiting and losing time rescheduling their own work.
This has, unfortunately, led to a myth that DevOps teams choose to ignore security. In reality, developers are keen to know that their apps and the environment they work in are secure—but at the same time, they don’t want security to get in the way of them quickly delivering valuable new products and software features.
So, is there a way for DevOps teams—one of the most important resources in many modern businesses—to embrace security without impacting agility? Can the integration of DevOps and security be done in a way that alleviates tensions and promotes collaboration, while actually improving both security and agility in the process?
Yes. The secret is automation.
Reconciliation Through Automation
As C-suite executives are now more likely to focus on security, due to the obvious financial and reputational consequences of a breach, DevOps teams should define how they protect and secure their multiple projects and production environments. Automating security as part of the CI/CD process allows DevOps teams to easily follow company security policies because they will be embedded into the automation pipeline.
This process can remain running with little concern, effectively minimizing stress about security. This still automates policy changes and activities so that there is a significantly reduced chance of error. Although the automation solution remains hidden, it can still be utilized at any point to view data on the vulnerabilities, compliance requirements, security policies and network connectivity, via its continuous scanning abilities.
Additionally, DevOps teams are already familiar with automated tools in their daily operations and communications—and they are likely to be accepting of switching to a security solution that integrates with their existing processes.
Automation is the key to creating reliable, effective and connected “DevSecOps” teams, as it makes the secure option the easy option. It combines DevOps’ existing use of automated tools to achieve their ultimate goal of continuous, on-time and on-budget deployments with security’s focus of reducing human error and maintaining continuous visibility into potential vulnerabilities.
Encouraging Adoption
A guiding principle of DevOps is collaboration, which is often equated with the idea of shared responsibility. To successfully embed security into the DevOps process, security teams and developers must work together and establish shared responsibility. But how?
Some organizations may assign a security representative in each development team. This person acts as a pivotal link between the two teams, improving communication and building a balanced process that considers everyone’s mutual interests. A continuous flow of knowledge-sharing among both teams ensures a level of maturity that allows a business to secure applications and services with an automated solution.
Security teams can begin to define “guardrail policies” that allow development teams to deploy continuously, with the caveat of having to obey security and compliance policies. This is critical for both teams. This new way of working means developers will be able to test their security posture at every step in the CI/CD pipeline and correct things when necessary, and security teams can comprehensively ensure security and compliance throughout the development process.
Embracing Collaboration
Any belief that there is common discord between DevOps and IT security teams is unfounded. While it cannot be denied that both teams affect each other, this is not due to conflict—it’s due to business needs. If the two teams work together, they can both achieve their goals and be part of a secure, innovative and profitable organization. The first step is to accept collaboration is a necessity and by embracing security instead of being concerned by it, DevOps teams can stay in control of how their needs work around IT teams’ processes. Then, an automated security solution can be deployed to improve the efficiency and outcomes of both departments—and, in turn, the entire organization. It’s time for DevOps to embrace DevSecOps.
Don’t fall for the marketing jargon: Running legacy infrastructure on a cloud instance is not actually cloud computing. Unfortunately, “cloudwashing” is common among enterprises as they attempt to jump on the cloud trend.
So, how do we dispel the hype to distinguish pure cloud software as a service (SaaS), platform as a service (PaaS) and infrastructure as a service (IaaS)? In this article, we’ll define cloud washed and cloud native, citing NIST standard definitions and industry expert opinions. We’ll add some cues to help decipher cloud washed versus cloud native architecture as you search for new tech. No whitewashing here; prepare for some hard truths.
What is Cloud Washing?
In recent years, services that simply work over the internet were rebranded with the “cloud” buzzword, perpetuating a tenuous, vague perception of the cloud.
Cloudwashed solutions are legacy, on-premises software held in a virtualized data center and rebranded as cloud software. Such tools originally were not built for the cloud and do not satisfy the NIST definition of true cloud computing.
While non-local hosting can reduce maintenance and server costs, by adopting cloudwashed services you sacrifice the benefits of cloud native. As Clint Boulton, writing for CIO describes:
“… the implementation doesn’t provide automated, continuous upgrades, let alone the multitenancy, self-service, auto-scaling and usage-metering capabilities that have come to characterize cloud services …”
“Cloud computing is a model for enabling ubiquitous, convenient, on-demand network access to a shared pool of configurable computing resources (e.g., networks, servers, storage, applications and services) that can be rapidly provisioned and released with minimal management effort or service provider interaction.”
NIST also defines three service models of cloud computing (SaaS, PaaS and IaaS), and specifies four types of deployment models (private, community, public and hybrid).
As you can see, the NIST definition of cloud computing is specific. Authentic cloud services are configurable, providing hands-on optimization. They are elastic, able to provision and support multiple application requirements. Such cloud architectures are able to efficiently scale to the unique computing workloads of various applications.
Furthermore, resources for cloud services are pooled. According to HiTechNectar, “True cloud platforms are built on a true multi-tenant model, with different physical and virtual resources dynamically assigned and reassigned according to consumer demand.”
A cloud-native application (NCA) is thus an application specifically designed for the cloud that satisfies the definition of cloud computing. A true cloud solution would be a proprietary service built on a platform such as CloudStack or OpenStack. Google Cloud, Amazon Web Services and Microsoft Azure are all examples of authentic cloud providers.
Cloudwashing Warning Signs
Cloudwashing is rampant in the ERP realm. As HiTechNectar reports, “Many enterprise software vendors make claims that their applications are built for the cloud when they are not.”
Thus, to avoid cloudwashed software, take heed of these warning signs:
Overuse of “cloud” in vague marketing jargon.
The need for custom integrations (this is against self-service).
No resource pooling.
Lack of developer tooling.
Lack of configurability.
Tooling not planned with redundancy in mind.
Server arrangement is not distributed.
Good Signs of a True Cloud
Thankfully, finding true cloud computing is made possible with the help from NIST, which lists five essential elements of cloud computing:
On-demand self-service.
Broad network access.
Resource pooling.
Rapid elasticity or expansion.
Measured service.
Another good sign is the vendor’s involvement in the Cloud Native Foundation. Companies also dedicated to open source projects tend to be more progressive and transparent.
Dissipating the Foggy Cloud
A recent CompTIA Report found that half of the companies in 2018 have 31 percent to 60 percent of their IT systems in the cloud. Cloud computing is still driving the rise of cutting-edge trends including AI, blockchain and IoT.
Unfortunately, though, not all purported cloud services are authentic. Ruled by an incessant need to stay relevant, tech marketing adopts and overuses certain jargon, and the “cloud” is not immune. Only by understanding the dichotomy between cloud washed and cloud computing can CIOs make educated decisions regarding their tooling choices.
The NIST cloud computing criteria is valuable because NIST is an impartial standards agency. NIST computer scientist Peter Mell stresses the importance of evaporating this fog to reveal the truth:
“… by adopting an authentic cloud, [businesses] are more likely to reap the promised benefits of cloud—cost savings, energy savings, rapid deployment and customer empowerment.”
To avoid being cloudwashed, seek out multi-tenant architectures with self-provisioning, pay-per-use models, virtual instances and linear scalability. Only then will you reap the rewards of an authentic cloud-native environment.
