devopsy.kinsta.cloud

Category: DevOps Toolbox

  • What Data Catalogs Don’t Reveal

    What Data Catalogs Don’t Reveal

    In today’s data intensive world, data catalogs, as the name implies, are simply catalogs of information (some human-entered and some extracted from metadata in systems). The importance of data in the enterprise has grown, and analytics plays an increasingly important role in generating value. So, while it’s important to collect and store and organize all this data, data catalogs are missing a major piece of the puzzle – they fail to draw accurate relationships between data assets and how they relate to (or are calculated from) each other.

    It is like having a catalog that contains a description of all the electronics components in the world, but without an understanding of how to assemble those components into a clock, a radio or even a computer. Or how to diagnose what needs replacing when the device stops working.

    Data catalogs cannot answer a whole host of questions along the lines of, “What are the consequences if I make a change to this dataset?” Even if the information in the data catalog is religiously updated, if a change was, in fact, made, users impacted by that change, whether directly and indirectly, have no way of knowing that a change was made. As a result, risk of unidentified issues is high.

    The table below summarizes the strengths and weaknesses of traditional data catalog products.

    Issue

    Data Catalog Strengths

    Data Catalog Weaknesses

    Data Discovery
    • Collect metadata information from a wide variety of systems
    • Give users flexibility to manually update information
    • Simplify searching for data assets across a wide variety of systems
    • Make it easy to identify the subject matter expert (SME) related to a dataset
    • Rely on manually maintained documentation
    • Do not automatically relate data sets to each other
    • Do not understand how data assets are calculated from each other
    • Fail to identify who uses a particular dataset (and how they use it)

    Data Governance
    • Provide a robust framework for managing the availability, usability, integrity and security of the data
    • Rely on manually maintained user-defined rules for dictating data access
    • Cannot automatically propagate access rules from one dataset to antecedent datasets

    Infosec Compliance
    • Make it easy for users to label personally identifiable information (PII) and other infosec risk data sets
    • Do not automatically infer the implications of data processing on the movement of PII inside the enterprise

    Data Quality
    • Data catalogs do not actually inspect data – so they do not assist with data quality issues
    • Complete failure to identify issues and communicate who those issues impact downstream from a data quality issue

     

    Data Discovery

    Data catalog solutions provide powerful frameworks for collecting metadata and facilitating documentation of datasets. The collection and aggregation of metadata is automated, and data engineers/analysts are, hopefully (but rarely), compliant in documenting their work so others can leverage it.

    Unfortunately, data catalogs fail to identify how datasets and processing rely on each other. There is no way to reliably identify which users access and rely on particular datasets, both directly and indirectly.

    Tasked with this objective, a data engineer can manually answer these questions by searching and analyzing the relevant data processing code. This approach uses expensive data engineering resources, takes time and results in reduced analyst and data engineering productivity.

    Data Governance

    When new data sets are created, the authors must tag the data to control data governance. Not only does this rely on compliance from the author, but the rules must be propagated to determine who can use the newly processed data. How should the rules be propagated? This is a complex issue. For example, the city in an address may be restricted, but aggregates across that city may not be. Defining these rules, in and of themselves, is complex — applying them is even more complex.

    Reliance on data engineers and analysts to comply with data governance documentation and systems for enforcement is error-prone and expensive. Sensitive datasets are at high risk of being exposed.

    Infosec Compliance

    Current solutions to information security compliance rely on:

    • Users manually documenting if their data processing activities use (or produce) infosec-sensitive data.
    • Automated identification of infosec-sensitive data by inspecting the actual data and using algorithms to identify how the data needs to be secured.

    Data catalogs are used to facilitate user documentation of data processing activities to comply with infosec compliance requirements. This process relies on humans manually maintaining documentation and systems (and performing actual data inspection.) These processes present significant security risks in and of themselves, because both data inspection – which itself presents a risk – and skilled manual work is required.

    Data Quality

    Data engineering already has processes to identify when there is an issue landing data in a table. Automated systems are available that inspect data values and make intelligent decisions (some even based on machine learning) about whether or not the data is acceptable. These systems exist outside of traditional data catalogs.

    The question is, who will that problem impact? You must identify who are the immediate users of the data. You must also identify the indirect users of the poor-quality data.

    The current approach to this problem is to have skilled data engineers review all data processing code and manually identify users to warn – this is both expensive and time consuming. Beyond this, the risk is that a user is not identified; what if the error impacts a report that lands on the CFO’s desk and they are not warned?

    Data catalog solutions are good at solving the problem for which they were initially designed – finding datasets that are of interest to data engineers and data analysts.

    The risk in DevOps is that data catalogs are relied upon for data discovery, data governance, infosec compliance and data quality issues that they fail to address in today’s complex data processing environments.

  • Consider Telemetry When Rearchitecting Applications

    Consider Telemetry When Rearchitecting Applications

    Australia and New Zealand have seen a big move to the public cloud, and “… when I talk to customers, I see huge interest in rearchitecting applications rather than doing a lift-and-shift right into public cloud,” said Scott van Kalken, F5 ANZ solution architect for DevOps and modern applications.

    That migration to the public cloud has led to massive interest in telemetry. When you go from a monolithic application to a distributed app, there may be 100 or even 1,000 APIs involved. Without telemetry, you don’t know what’s going on with those APIs. But the telemetry needs to be consistent.

    F5’s 2021 State of Application Strategy report found 75% of respondents considered telemetry, especially about application security and delivery, important for meeting business outcomes, and 95% of respondents said they are missing insights from their existing monitoring and analytics solutions.

    Van Kalken recounted a conversation with a practitioner as an example. This practitioner gets yelled at when problems arise with partner APIs used by the application he manages. So, even though these APIs are not technically his responsibility, he needs visibility into their status. Only by combining internal telemetry with telemetry from these partner APIs can he make sense of what’s happening within the application ecosystem.

    And it’s not just a question of whether an API is available or not. Problems can arise when it is responding slowly. “‘Is that normal? Is that normal for this time of day?’ I’ve got to have some sort of telemetry data available to me to make decisions,” van Kalken said.

    Enabling Security Automation with Telemetry

    An aspect that’s particularly interesting to him is the ability to automate or otherwise take action based on security telemetry data.

    A simple example is a security event that reveals some kind of systems abuse. Automation can add a firewall rule to block the associated traffic.

    As organizations increasingly adopt event-driven architectures, events and responses occur at the application, network, infrastructure and other levels. “The minute that you automate those event-driven architectures with security, it opens up a world of possibilities.”

    It’s not just about automating security. Imagine an API gateway that serves a variety of APIs to customers. Telemetry might reveal that two of the APIs are starting to slow down; in which case, “I might kick off a workflow that just deploys more of them so that we can handle the load. But if you don’t have that telemetry, you can’t do that,” he said.

    In practice, this involves telemetry, analytics and automation.

    “When we show [customers] how you can make use of the analytics in a really easy way, their eyes light up,” van Kalken said.

    Multi-Cloud Migration

    A further consideration is that rearchitecting applications isn’t just about splitting monoliths into smaller pieces and redeploying them as microservices. There’s also a desire to run them across multiple public clouds to reduce risk, and also to deploy applications at the edge where they are closer to their users.

    The 2021 State of Application Strategy reported that just over three-quarters of respondents are already using or have plans to use the edge to capture benefits related to application deployment, performance and data availability.

    “This is really important, because as an application developer, I want to deploy my application once and I want to say, ‘I want this distributed to all of these places around the globe’,” van Kalken said.

    But if the application is deployed in, say, five locations, then from a telemetry perspective, it is five applications, not one.

    “I’ve got five sets of telemetry, because I’ve effectively got five different applications deployed out there. It’s the same app, but it’s in five locations, and the consumers are different in every location; the way that they interact is different and the telemetry that I’m getting is going to be different,” he said. “So, I need to actually suck up all of that telemetry, and make it meaningful in some way,” he explained.

    Telemetry Enables Clearer Vision

    “A distributed cloud architecture is right for these modern applications. We recently bought a company called Volterra, and this is one of the areas that they play in — distributed applications, and specifically, abstracting multi-cloud and multi-cluster Kubernetes application deployments,” he said. This area of focus will be critical in the future, van Kalken said.

    He predicts that as organizations move to 5G, applications will become increasingly distributed. If that does happen, it will be extremely important to receive telemetry from all components of the application so it can be analyzed and used to support intelligent decisions, as part of the distributed application deployment.

  • OpsRamp Extends AIOps Reach to Embrace Prometheus

    OpsRamp Extends AIOps Reach to Embrace Prometheus

    OpsRamp today announced it has added the ability to ingest data captured via the open source Prometheus monitoring software into its namesake IT operations platform infused with machine learning algorithms.

    In addition, the company is adding support for customizable dashboards using the Prometheus Query Language to create visualizations of hybrid infrastructure performance. Also, a set of available tools makes it easier for IT teams to discover when workloads have been added to a cloud. These capabilities became available with the Spring 2021 Release of OpsRamp’s IT operations platform infused with artificial intelligence (AIOps).

    Finally, OpsRamp has also created an alert definition model that streamlines the alerts generated by the AIOps platform and its integrations for the Microsoft Azure cloud and the Cisco HyperFlex platform deployed in on-premises IT environments.

    Deepak Jannu, senior director of product marketing for OpsRamp, said that as IT becomes more complex to manage, many IT teams are starting to realize that managing these environments without help from an AIOps platform won’t be possible. As such, much of the fear and loathing that was initially associated with AIOps is giving way to acceptance as IT teams start to realize these platforms augment their capabilities versus replace them, added Jannu.

    Much of the complexity is being driven by the rise of cloud-native, microservices-based applications that tend to have lots of dependencies. OpsRamp has primarily been encountering Prometheus in these environments. However, Prometheus can also collect data from legacy applications, as well, so over time, the expectation is the open source monitoring tool will be employed more widely.

    As AIOps platforms become more widely employed, there is no doubt job roles within IT organizations will change. As lower-level tasks become automated, IT teams should have more time to spend resolving complex issues once the AIOps platform has identified a baseline for how applications should optimally behave. In many cases, the need for personnel to perform rote tasks that no one in IT wants to actually do will be eliminated altogether.

    DevOps teams that are committed to automating IT tasks will naturally be among the first to employ an AIOps platform to augment the capabilities of site reliability engineers (SREs) that are often challenging to hire and retain.

    In general, Jannu noted that the observability that many IT organizations now demand can’t be achieved by simply collecting more metrics. IT organizations will need to invest in an AIOps platform capable of making sense of all the instrumentation data being collected. The challenge now will be defining the right division of labor between man and machine in an IT environment, Jannu added.

    At this juncture, AIOps, to one degree or another, is inevitable. In many cases, machine learning algorithms are already being employed to optimize applications or a specific class of hardware. The decision IT teams will ultimately need to make is determining whether to replace their existing IT operations platform or wait for the provider of their existing platform to get around to adding AI capabilities.

  • ServiceNow Acquires Lightstep to Gain Observability Platform

    ServiceNow Acquires Lightstep to Gain Observability Platform

    ServiceNow today announced it has acquired Lightstep as part of an effort to add an observability platform to its IT operations portfolio. Terms of the deal, expected to close this quarter, were not disclosed.

    The Change Intelligence observability platform from Lightstep is based on a time-series database that is capable of processing a trillion events each day. It was built by the same team that created Monarch, the globally distributed in-memory time series database system that Google employs to monitor its applications and systems. Lightstep team members also co-created both the OpenTracing and OpenTelemetry open source projects for collecting traces, metrics and logs.

    Pablo Stern, senior vice president and general manager for IT workflow products at ServiceNow, said ServiceNow will employ Lightstep to enable IT organizations to observe both monolithic and microservices-based applications as IT operations and DevOps teams continue to collaborate more closely with one another. The goal is to provide a single pane of glass through which IT teams can observe their entire IT environment via the Now platform that ServiceNow makes available via a software-as-service (SaaS) delivery model.

    That approach will enable DevOps teams to programmatically manage IT environments alongside IT administrators employing graphical tools, noted Stern.

    Lightstep CEO Ben Sigelman added that, in time, the number of users of that observability will only increase as more application stakeholders start to track software development projects that are at the heart of various digital business transformation initiatives.

    ServiceNow

    Thanks mainly to the rise of microservices that are now being deployed across highly distributed computing environments, it’s become more difficult to discover the root cause of an IT issue. Microservices-based applications have lots of dependencies that, while serving to make the overall application more resilient, conspire to make it difficult to determine which specific microservices might be adversely impacting the performance of an application.

    The time-series database that lies underneath the Change Intelligence observability platform makes it feasible to go beyond logs and metrics to process distributed traces that provide more insight into application behavior. Ultimately, the goal is to enable organizations to innovate faster with a higher degree of confidence in the IT environment, said Sigelman.

    As far as the march to observability in the enterprise is concerned, it’s still early days. While observability may be a core tenet of any DevOps practice, most IT organizations rely on myriad tools that surface data the teams then attempt to correlate. Whenever there is a major issue, those IT teams typically convene in a “war room” to compare reports generated by the various IT monitoring tools employed. Observability platforms that collect and process distributed traces used to instrument an application using a consistent set of specifications, frameworks and libraries. The OpenTracing and OpenTelemetry tools are significant because they provide an open source approach to instrumenting applications that reduces the total cost of observability. In comparison, many legacy application performance management (APM) platforms employ proprietary agents to instrument applications.

    Naturally, each IT organizations will be moving down the path toward an observability platform that provides greater application context. At this point, however, it’s not clear to what degree observability platforms will need to be installed before making the transition to microservices versus being a capability that only gets appreciated once the trouble with microservices finally becomes apparent.

  • Survey Illuminates Long Dark Road to Hybrid Cloud

    Survey Illuminates Long Dark Road to Hybrid Cloud

    While hybrid cloud computing will play a critical role in enabling digital business transformation it turns out most IT organizations lack the skills and technologies required to achieve it.

    A survey of 108 IT executives conducted by the research firm Pulse on behalf of CloudBolt Software, a provider of a cloud management platform, finds 94% believe hybrid cloud computing will be critical to digital transformation. A total of 92% also noted that empowering end users through self-service IT is a digital transformation accelerator, while 99% said successful automation depends on streamlining integrations.

    However, 78% of respondents said they lack the visibility necessary to optimize cloud deployments. More than half (56%) believe that their self-service offerings fail to meet user needs and more than three-quarters (76%) still rely on some form of custom coding when it comes to integration.

    In terms of the capabilities respondents are looking to acquire, 80% noted they want real-time insight into cost optimization and security remediation. A total of 71% want self-service to be easy for end users, while 62% want an approach to integration that doesn’t require deep domain knowledge or special expertise.

    CloudBolt CEO Jeff Kukowski said the biggest immediate obstacle when it comes to hybrid cloud computing is the lack of visibility IT organizations have into their existing IT environments. IT teams are not going to be able to unify the management of distributed IT environments without it, much less achieve anything approach observability, noted Kukowski.

    Pressure on IT organizations to accelerate application development while simultaneously reducing costs rose as the economic downturn brought on by the COVID-19 pandemic began. The challenge is that each new platform adopted by an organization tends to increase the total cost of IT as organization acquire tools and hire specialists to manage those platforms. It’s not likely all those platforms will be managed via a single console, but Kukowski said it’s clear IT organizations are moving toward a more federated approach that will enable them to centralize the management of IT.

    In the meantime, Kukowski noted organization are collectively wasting billions of dollars on IT platforms that are employed inefficiently. Most of that waste stems from either overprovisioning of IT infrastructure, or simply because someone didn’t think to turn off a virtual machine left running on a cloud service.

    It may be a while before IT organizations are able to strike that perfect balance between flexibility and control that currently remains elusive. However, organizations that focus on improving a few small things today will find themselves in a much better position to manage what is rapidly becoming an over extended enterprise tomorrow. Right now, Kukowski estimated only about a quarter of IT organizations have the sophistication required to manage a hybrid cloud computing environment.

    There’s no doubt the management of IT will become more automated in the months and years ahead. The challenge organizations face today is determining what tasks either can or soon will be automated. Once that’s determined, it becomes a lot easier to identify what DevOps skills those IT teams will require tomorrow.

  • DevOps Practices Gap Assessment

    DevOps Practices Gap Assessment

    Using DevOps to continuously deliver software has proven value for organizations that are on a path towards digital transformation and continuous improvement. To get the maximum value from DevOps, it is important to use recommended best practices, inclusive of people, processes and technologies. A gap assessment is a great way to efficiently evaluate an organization’s practices for DevOps, and determine a strategy for improvement. Gap analysis provides valuable inputs for formulating a strategy and roadmap to improve a topic including DevOps.

    My SlideShare, Gap Survey, Assessment and Analysis For DevOps, explains how my Nine Pillars Gap Assessment Tool greatly speeds up the gap survey, gap assessment and gap analysis process for DevOps.

    It is important to clarify a few definitions for this blog, because I have found that different people and organizations use these terms differently. In this blog, and in my consulting work, I use the following definitions.

    • Gap survey is a discovery tool used to collect information about the current state of practices for a topic.
    • Gap assessment is a process to determine differences (gaps) between current practices and recommended practices for a topic. Gap assessment is also used to refer to the entire collection of activities in this list.
    • Gap assessment workshop is a meeting in which the gap assessment is validated prior to gap analysis.
    • Gap analysis is the process of determining priorities for reducing gaps found during a gap assessment.
    • Gap assessment tool is software that helps perform gap surveys, gap assessments and gap analysis.

    For those that want to dig deeper on this topic, additional definitions and contextual information related to the way I do gap assessments are clarified in my book Engineering DevOps.

    I define a seven-step process for gap assessments. The steps are as follows:

    1. Pick a topic to analyze. The same processes and tools can be applied equally well to almost any topic.
    2. Determine recommended practices for the topic. Something that is worth doing a gap assessment for, such as DevOps, will normally be fairly complex with dozens, if not hundreds of practices. Categorize the practices into groupings (I call the groups “pillars”) to make the list of practices more understandable and meaningful. To do this will require some research and work. For example, I have a database of practices and pillars for DevOps, continuous testing, DevSecOps and SRE and some other topics such as DevOps.
    3. Enter the practices from step two into a gap assessment tool. The tool can be a simple survey tool you create, a commercial survey tool or something custom-made.
    4. Determine who needs to be included in the gap survey.
    5. Collect data using the gap survey.
    6. Perform a gap assessment.
    7. Perform a gap analysis.

    For the remainder of this blog, I explain the above seven steps for DevOps.

    Step one: Pick a Topic to Analyze
    In this blog, the topic is DevOps.

    Step two: Determine Recommended Practices for DevOps
    For an example DevOps assessment, I used the practices under the nine practices categories described in a blog that I co-authored called Nine Pillars of DevOps Best Practices. The nine practice category pillars are:

    • Leadership
    • Collaborative culture
    • Design for DevOps
    • Continuous integration
    • Continuous testing
    • Continuous monitoring
    • Elastic infrastructure
    • Continuous delivery
    • Continuous security

    Step three: Enter the Practices from Step Two into a Gap Assessment Tool
    A gap assessment tool, pre-loaded with a sample of DevOps practices, in a file called DevOps Assessment, can be downloaded for free from one of the resource pages found on my website Engineering DevOps. You can edit the practices categories and add/delete practices from each category if you prefer to make changes.

    Step four: Determine Who Needs to be Included in the Gap Survey
    For the gap analysis to be comprehensive, people in roles that are affected by DevOps practices need to be surveyed, or at least represented, to ensure their perspectives are included. The following are example roles that typically are included.

    • Business leaders – because they influence culture and budgets for DevOps.
    • Developers – because they need to design in accordance with DevOps practices.
    • Project owners – because they influence product work priorities.
    • QA testers – because they need to test in accordance with DevOps practices.
    • Ops – because DevOps practices affect operations.
    • Security – because DevOps affects security practices.

    The survey can be conducted for an individual application, a group of applications, or all the applications in the enterprise. However, it is important that the gap assessment and gap analysis be performed on the organization segment that is being targeted for improvement.

    Step five: Collect Data Using the Gap Survey
    A gap survey should allow each surveyed person to enter an importance level score, a practice level score and comments for each practice. All this information is essential for the gap assessment and gap analysis. In the gap survey included within my gap assessment tool, survey respondents are asked to score the importance of each practice as one of 0=not relevant, 1=not important, 2=nice to Have, 3=important, 4=very important or 5=critical. Practice level choices are 0=not sure, 1=rarely, if ever, 2= sometimes, 3=most of the time, 4=always or 5=We are really good at this. Comments that are relevant to qualify or explain the scores entered for each practice should also be entered, especially if there is any doubt or ambiguity.

    Step six: Perform a Gap Assessment
    The gap assessment process requires all the practices scores collected from the surveys to be collected and assembled to calculate an aggregate set of scores. Gap scores are calculated using a formula that weights each practice level score with the corresponding importance level score. A visual representation helps to identify practices areas and individual practices that have the highest, most important gaps.

    No matter how professionally written the practices and score definitions are, it is not unusual for some people to misunderstand and to enter scores that they would not have otherwise. For this reason, it is important to ensure that the data collected is validated before conducting the gap analysis. The preferred approach is to conduct a gap assessment workshop with key representatives from each role that participated in the survey. During the workshop, the scores for each practice that have a high deviation between survey responses are discussed and, if necessary, adjusted.

    Step seven: Perform a Gap Analysis
    The gap analysis process involves extracting the high gap practices and tagging and ranking each of them against solution categories determined by a consultant or topic expert.

    The results of the gap analysis indicate where solution strategies and implementation roadmaps need to be focused to reduce the most important gaps.

    What This Means

    While it is true you can easily stand up DevOps tools quite quickly, this alone will not realize the benefits of following effective and safe DevOps practices. Those who are serious about DevOps will do well to learn from others, and be serious about using recommended DevOps practices. This blog explained a gap assessment approach, which uses my nine pillars of best practices for DevOps. My free DevOps gap assessment tool facilitates getting a gap analysis for DevOps and priorities to improve DevOps practices for an organization.

  • Data Needs and Challenges with CData

    Data Needs and Challenges with CData

    The data businesses and applications need are everywhere and access through a multitude of technical interfaces, including APIs, database queries, data layers like ODBC, cloud services and SaaS applications (to name a few).

    In this episode of TechStrong TV, Jerod Johnson, technical evangelist at CData, joins Mitch Ashley to discuss the challenges of meeting the data needs applications require to support digital and transformation initiatives.

    The video is immediately below, followed by the transcript of the conversation. Enjoy!

    Transcript

    Mitch Ashley: I’m joined today by Jerod Johnson. Jerrod is technology evangelist with CData. Jerod, good to be talking to you today.

    Jerod Johnson: Yeah, no, it’s my pleasure to be here.

    Ashley: Well, as an old ex-DBA, data is always kind of a passion of mine, interest of mine, so I’m anxious to hear more about CData. Would you introduce yourself and tell us a little bit about what CData does?

    Johnson: Yeah, sure. So, like you introduced me, my name is Jerod Johnson, I’m a Technology Evangelist for CData Software. I’ll start with CData. So, we call ourselves a leading provider of data connectivity solutions. In short, what that means is, we make it easier for every user in an organization to connect to their data, which lets them focus on the job of finding value from that data or building applications on top of that data instead of having to focus on the integration, instead of tasking IT teams to build connectors or pipes or flows, whatever the case is, in order to work with the data that they need to drive business.

    Ashley: Mm-hmm. You are number two on my list of what are the reasons people come to IT when they’re building apps.

    Johnson: Yeah.

    Ashley: And that is, security, data, and other apps.

    Johnson: Yeah. [Laughter] 

    Ashley: So, always need data. Well, tell us a little bit about yourself. I’m curious—how did you become a technology evangelist? How did you—

    Johnson: Yeah, so, I actually spent six years teaching before I got into software. So, my education background was in teaching and computer science, and spent six years in a classroom teaching math and computer science, and just decided that we needed a change, for myself and my family.

    So, I started at CData, I was actually a member of the support team, which meant I was answering e-mails and helping solve customer problems. And then I moved into engineering, and spent some time there, but it really became obvious that there was a need in the company for somebody with a skill set that I had, being able to speak to people, being able to present information, even write technical documents. 

    And so, I was pulled into the marketing team, and I’ve been doing that since 2016 for the company. So, writing technical documents, giving talks at trade shows when those used to happen, through webinars.

    Ashley: When we were [Cross talk], right? Hopefully soon, hopefully soon.

    Johnson: Yeah.

    Ashley: Interesting. You’re not the first person I’ve talked to who’s come out of an education background and moved into technology and, in your case, moving into an evangelist role. That whole skill of being able to explain sometimes very complex concepts, sometimes not so complex, but you may not necessarily have the context or the background for a certain thing that you need to learn or understand—

    Johnson: Sure.

    Ashley: Or even explain to non-technical folks, right? So, a great thing to have, especially in the DevRel world that we all live in now of really focusing on how we work with service and kinda think like a developer.

    Johnson: Sure.

    Ashley: Because that’s the people who are using it, whether they’re business developers or people inside IT. Let’s talk about sort of the data challenges, and I’m curious, maybe you’ve seen some things over the last kinda year plus, too, with the acceleration of digital transformation projects.

    Johnson: Yeah.

    Ashley: It seems that one of the challenges always is data sprawl—it’s everywhere, and every application is creating data, consuming data, sometimes creating data exhaust that’s unused or somebody else might be able to use it. It’s no small feat just to find where it is, more or less find out how you can get to it and use it.

    Johnson: Yeah, no, absolutely. I think there have been studies done, I think programmable web is probably the best place to look if you’re looking for a list of APIs. And I think that they have 22,000 public APIs listed at this point.

    Ashley: Wow.

    Johnson: So, I mean, and each of those APIs, you know, there’s data behind it. So, you have that kind of explosion, the API explosion. On top of that, you’ve got private APIs and then databases, data warehouses, these traditional stores. So, data is everywhere.

    One of the things that we’ve found is that, as data explodes, as businesses transform digitally, you get the opportunity to pick the best service for your needs. So, you know, you’ve picked your CRM application because it meets your needs the best. You’ve picked your ERP solution because it meets your needs the best.

    The consequence of that is that your data is now spread everywhere. So, your CRM data is somewhere, your ERP data is somewhere, your warehousing data is somewhere else. And so, there’s this, as you said, this data sprawl where data is everywhere.

    And what we do is, we make it all look the same. So, we like to say that CData lets you see the world as a database. So, we’ve created technologies that allow users to connect to their data, no matter where it is, as if they were connecting to a database. And so, I’ll drop some buzz words, some acronyms here—so, JDBC, ODBC, ADO.NET, these are all database connectivity standards that we’ve produced that now can connect to things that are not databases.

    And so, you know, what that does is, it really democratizes data access. Now, anybody can get to data, from any of the tools or applications that they’re using. So, whether you’re looking at an IT team or a DBA or something like a citizen analyst or a data scientist, it doesn’t matter. As soon as your data looks like a database, anybody can work with it.

    Ashley: Mm-hmm. That’s interesting you say that citizen data scientist—it’s not just developers. Of course, developers are very familiar with ODBC and other connectivity layers for data access, which are fantastic for developers, but you have to have some pretty specific development skills to be able to [Cross talk] this thing.

    Johnson: Sure.

    Ashley:  A lot of this data is sitting on a Dropbox file server in some cloud application, you know, we signed up for a CRM system—well, guess what, I need to connect that with my workflow for when an order comes in. And, you know, as good as Zapier and things like that are for doing kinda nice integrations for automation, that’s not even gonna cut close to solving these kinda problems.

    And you mentioned, too, making it look like a database. I’m curious, from my experience, it’s how you get to it and making that understandable.

    Johnson: Sure.

    Ashley: But it’s also understanding the data and what it means and how it’s represented, because a customer is not necessarily defining the attributes similar in a CRM that they might be in an order system, there may be some variations in just the format of how we capture that data.

    Johnson: Yeah. Yeah, so, you know, what we do is, everything that we make sits on top of the API. So, your API is gonna have endpoints, and those endpoints are gonna expose fields for entities.

    So, let’s take a customer, for example. When you hit a customer using one of our drivers—hit the customer, well, now, it’s a table, it was an API endpoint. What we’ve done is, we’ve kinda tabularized that data model. So, now, every customer is a row in a database table and then each of the fields for that customer becomes a column.

    And then, you know, what we do from there is basically free you to map the data from one system to another as you need, in the tool that you’re using. So, we don’t force a model on you, we’re not trying to create one universal picture of a customer across your organization. We’re gonna let your business specialists do that part. We just make it easier to see, and because it looks like a database, the schema, the metadata, however you define it is really self-describing. 

    Ashley:  Mm-hmm.

    Johnson: You know, if you can talk to a database, you can point your tool. So, let’s say you’re looking at something like Tableau. When you point it to a database, it’s gonna show you all of the tables that are available and then all of the columns that are available for each of those tables.

    Well, we do the same thing, but for something that isn’t a database. So, it could be a NoSQL store like MongoDB or Couchbase or even Amazon S3. It could be a SaaS application like NetSuite, or even Dynamic CRM. So, whatever it is, you get that full tabular model where you can see exactly what’s there and then you can decide how best to work with that data.

    Ashley:  I mean, think about the advantage of, if you don’t have, necessarily, all the programming skills you might want, to go get that data, just not having to use an ABI, but use the driver instead.

    Johnson: Right.

    Ashley: And so, just get that to me in a table form that I can—now, I can compare how one customer is _____ and one, I’m just making that example, to another customer set of attributes and other applications.

    Johnson: Yeah.

    Ashley: Sometimes it’s hard enough just defining who is—what a customer is. [Laughter] 

    Johnson: Sure.

    Ashley: I remember that modeling. But now, you can see what the data is without writing a whole bunch of software and translation and trying APIs and learning specifics about this particular service or this cloud application or whatever it might be.

    Johnson: Right. Yeah, yeah. And that’s one of the things that we really offer is, we free you from that integration problem. We free your developers, we free up IT resources. So, APIs are constantly changing. I don’t know if you’re familiar with the Facebook API, but I think they give a new version, like, every three minutes. I think it’s realistically like every three weeks, but it feels like three minutes when you’re a developer.

    Ashley:  I try not to be familiar with the Facebook API. [Laughter] 

    Johnson: Yeah, which—that’s fair. [Laughter] But with our drivers, our engineers are the ones that are keeping up with those changes. So, every quarter, we release a new product, and that includes all of the API changes that have happened on the back end. So, the only thing you have to do is literally install a new driver into your system. There are no changes—no breaking changes to the API, we merge everything as much as we can. And it really, as I said in the beginning, it lets your business experts focus on your business, whatever that is.

    Ashley: Mm-hmm. I’m curious, too, is it primarily developers that are using CData and the drivers that you have? How many folks may be from a non-super technical environment, maybe some maybe a little technical skills and might use CData?

    Johnson: Yeah, so, that’s a good question. I think it’s a pretty solid blend. So, you know, one customer story might be a mom and pop shop, they wanna be able to look at their QuickBooks data in Excel, they wanna be able to work with it, print it out, edit it, even, and update it that way. They don’t like the QuickBooks UI, they’re familiar with Excel, and so they simply install our Excel add-in, and now they’re connecting directly to QuickBooks from Excel, with full read and write capabilities.

    We can get more complicated than that with customer stories. We have one customer who was dealing with that data sprawl, the data silos where their data is everywhere, and they needed a solution that allowed them to replicate all of their data into the data warehouse that they were already using for BI and analytics. And so, we have a tool, CData Sync, that allows them to do that. So, connectivity—solving the connectivity problem and solving the replication problem, so they get automated, incremental replication of all of their business data into one place for BI and analytics.

    And then, you know, I touched on this before, but we solved that BI and analytics problem in other ways. So, you know, you have data scientists that are familiar with, that really love their reporting tools, whether that’s Tableau or Power BI or any other tool on the market. They don’t wanna have to change tools just to be able to connect to the data. And with our connectors, that’s exactly what they get to do. They get to stay in the tool that they’re already using.

    Ashley: Mm-hmm.

    Johnson: But when it comes to developers, this is where we kind of get into our most complex customers. Ultimately, the customer is an organization that’s building a data-centric tool or a platform, and their developers, you know, they’re the ones responsible for maintaining the connections to data from those tools and platforms. So, you know, you’re building the best BI tool on the planet but if it only connects to SQL server, is it really the best? So, we’ve had companies come to us and use our drivers to build products. And what it really frees up there is one-off API integration.

    So, if you’re building a tool and you’re connecting directly to the API from your tool, integrating against Salesforce looks one way, integrating against NetSuite looks another way. Integrating against Google BigQuery looks a completely different way. But as soon as you use our drivers, it all looks the same. If you can develop a tool that can talk to a database, you can develop a tool that can talk to anything.

    Ashley: And one of the situations I’ve run across several times is through M&A, when you start bringing companies together through acquisitions and of course—

    Johnson: Right.

    Ashley: – you know, you need information, you need data to operate. I mean, just imagine combining a sales organization across two or multiple companies together and ones in Salesforce and one’s in HubSpot and one’s in homegrown whatever.

    Johnson: Sure.

    Ashley: And, you know, getting that data quickly, you don’t wanna have to go through a whole conversion and, “Okay, now everybody’s in Salesforce” or whatever your tool of choice.

    Johnson: Right.

    Ashley:  Maybe that’s where you go, but you may live six months, two years, where you need data out of multiple systems to see your pipeline, see the sales bundle, see, you know, how we’re closing the deal and what’s the conversion rate and all those kinda things for. And tools like CData are extremely valuable in getting to data very quickly to get value out of that [Cross talk].

    Johnson: Right, yeah. I mean, that’s, legitimately, our own personal story. So, when we first started as a business, we had a homegrown CRM, and then we started incorporating HubSpot. And now, you know, as we grow, as we get bigger, those systems don’t work for us anymore, and so now, we’re using Salesforce. And we’re in a place right now where we are concurrently using all three, and actually using our own technology internally to build reports, to do analytics, and then, as part of that process, to migrate data from one system to another—so, moving our data from our internal CRM into HubSpot and then into Salesforce.

    So, because the drivers are standard based, because they make things look like a database, it’s incredibly empowering for IT teams and DBAs to work with the data and to either consolidate it or migrate it, or leave it where it is, but build up connections to it in a way that lets the other data experts, the C suite, whoever needs to see the data, see it quickly and easily.

    Ashley: That’s a good point. We don’t always wanna take on the work of combining things, migrating things. If you can leave data in place, that makes sense to do.

    Johnson: Yeah.

    Ashley:  Boy, what an asset, because now you can get to it, but—okay, just manage it the way you were. And it’s great that you could relate to that example that I brought up where we didn’t coordinate that, at all.

    Johnson: Yeah.

    Ashley: I’m curious, how about the security? What comes with data, security is always super important to access control, things like that. Where in the stack does that fall, and how does CData play in those roles?

    Johnson: Yeah, so, we will leverage whatever security is available from the API. So, you know, as one example, Salesforce users have different permissions, whether that’s permissions to access different entities, permissions to only read entities, to write entities, whatever the case is. 

    So, when you connect to Salesforce using any of our connectors, you choose the user that you wanna connect through and then everything that they can do, you can now do through our connector. Anything that they couldn’t do before, you can’t do. And then we support the full gamut of authentication protocols. So, if an API supports OWASP, we’ll support it. If an API supports other authentication schemes, even—so, like, your Microsoft based products, on premise, we do NTLM. If you’re looking at Kerberos authentication, we can manage single sign-on—just whatever is available there, we allow.

    And then the protocols that the API uses for security—so, SSL, the encryption, all of that is supported as well, so.

    Ashley: I think an advantage you have, since you’re not pulling the data out of those systems and storing it somewhere else, it’s still data that’s living there—

    Johnson: Right.

    Ashley: – all the security systems, the access controls permissions, policies, et cetera, are still within those applications, however they’re managed, either standalone or collectively across those. So, you’re not having to devise some new data permission scheme from that, right?

    Johnson: Right, right.

    Ashley: That’s gotta be a huge plus.

    Johnson: Yeah.

    Ashley: Cool. What do you see as—while we have some time left, here—I’m interested in what you see as sort of emerging applications. You know, we’ve seen a lot of acceleration with digital transformation.

    Johnson: Yeah.

    Ashley: Digital engagement with customers, automation—there’s many more things happening, and of course, those both need data and create data.

    Johnson: Yeah. So, you know, I think one of the things that we’ve seen of late is just this migration to the cloud, where everything is in the cloud, your data’s in the cloud, you’re doing BI in the cloud. So, through products like Tableau Online or the Power BI service, instead of Power BI Desktop, Google Data Studio, AWS QuickSight, whatever the case is, you know, you’re working with data from a cloud platform now instead of something on premise.

    So, what we’ve created is, we have a cloud platform that creates a cloud to cloud connection. A lot of the tools that exist today, you know, even if you wanna go from a cloud platform to a cloud data store, you’re still required to put a gateway on premise somewhere. You know, this is the case with a lot of the Power BI connectivity. And, you know, if that doesn’t—why would you go down from the cloud, to on prem, back up to the cloud in order to work with your cloud data? And so, that’s—the CData Connect Cloud Platform is what it’s called, but it allows live, real time cloud to cloud connectivity. 

    And so, you know, that’s one of the, kinda the emerging trends that we’ve seen is people wanting to just do everything in the cloud. And it makes sense, especially—you know, I think about this past year, right? Everybody’s working remote, you know, you don’t have an on premise server sitting somewhere that can run your platform, you have to move it to the cloud so that everybody that’s working from home can access it, and that’s just one example of this cloud migration that we’ve seen.

    Ashley: Nothing like a pandemic—I say that knowing the human tragedy side of it, all due respect, but—

    Johnson: Yes.

    Ashley:  – to really force some very aggressive changes in our business and how we operate and some really interesting things. Is there a particular area of, you mentioned cloud and more usage of cloud services—what’s sort of the next frontier, if you will, that CData is tackling of how to make this data problem easier?

    Johnson: Yeah, so, I think—you know, we’re thinking about that cloud integration problem and trying to look for ways to help that. But you know, an area that we’ve started pushing into recently is kinda low code developers. So, I had mentioned you have your IT teams and your full stack developers that are used to working with data, that are used to writing code. But something else that we’ve seen crop up is this low code application development—whether that’s in a cloud platform or an on premise platform. And it’s great. You can build a workflow or an app using a drag and drop menu. These platforms tend to be intuitive. And now, your citizen analysts can build a workflow to combine data to get what they need into their BI tool.

    The problem that we’ve seen there is connectivity to data from these platforms, and so, that’s kind of an area that we’ve been pushing in an integration kind of marketing side is reaching out and trying to connect with low code and shoring up our integration with these platforms.

    Ashley: Yeah, and by the way, that happens to be an area of research that we’re working on with CData, and we’ll talk a little bit more about that here coming up in a few weeks—some really good, interesting findings from that research.

    And that was really interesting to Accelerated Strategies, the analyst firm that I run, because there is so much interesting in low code/no code right now.

    Johnson: Yeah.

    Ashley: And of course, it’s so broad of what that means. It could mean a spreadsheet to somebody, you know, as well as low code or platform, you know, there’s a number of platforms for creating applications and accessing data through a CData kinda combined platform. So, it’s really interesting, to me, of kinda who is using low code technologies to create applications.

    And I think the world used to view it, or maybe currently views it as, those are the business users, non-technical. That’s certainly a population, but it’s more and more coming into IT is what I’m seeing.

    Johnson: Yeah, that’s cool. I mean, as somebody who was a developer, was a support engineer, it’s fascinating for me to get my hands on a low code platform and figure out, you know—well, what exactly can I do with this?

    Ashley: Mm-hmm. Yeah, it’s interesting technology. Another way to solve problems, another technology, right, to build the applications.

    Johnson: Right.

    Ashley: How about the mobile side of this? With so many more mobile apps, we’re almost in an era, maybe we’re in the era of, “I’d like fries with that, I’d like a mobile app with that.”

    Johnson: Sure.

    Ashley: Or maybe it is mobile first. Does that change the dynamics of some of the data needs or challenges?

    Johnson: So, I think once you start looking at mobile applications, you’re looking for lightweight applications, right? You can’t—I mean, phones are impressive now, right? Like, I think about the amount of data that I can store on my phone versus the first smartphone that I got, you know, the amount of space that I have for apps. But at the same time, you know, embedding a software library or a driver or whatever the case is into your app, it doesn’t always make sense. Especially if you’re building an app that supposed to be able to connect to hundreds of different data sources.

    Ashley: Mm-hmm.

    Johnson: You know, are you gonna install 100 different 7 megabyte drivers with your app? Like, is that really the best way to go? And that’s another place where our cloud platform makes sense. So, you can be on your mobile app, you can connect—you can build an application that connects to a cloud platform that then allows users to connect to data from anywhere.

    Ashley: Mm-hmm.

    Johnson: So, you know, using the same technology that we have, it’s just consolidated into one single platform. So, you hit that platform and now, you know, you can pull in Salesforce data, you can write back to HubSpot, you can dump a report as CSV into AWS S3, all through one platform.

    Ashley: And you know, so many of those apps aren’t necessarily greenfield apps, right? All that data, maybe even the transactions processing, much of it is happening in other apps, other cloud locations.

    Johnson: Right.

    Ashley: So, when you’re building a mobile app, you know, you’re not inventing everything from the bottom up, right?

    Johnson: Right.

    Ashley: You’re merging a lot of existing resources—functionality as well as data. So, it seems like no better—what better of a kind of application where making it easier to get the data? Because, you know, mobile developers don’t like writing database access code, right? They like building interfaces and doing cool interactions and useful interactions with customers.

    Johnson: Right, right.

    Ashley: So, when you can let them focus on that and really create great experiences with that data, that’s a win/win for everybody, it seems like.

    Johnson: Yeah, no, I agree.

    Ashley: Well, cool. I really enjoyed talking with you, and where can folks find out about CData? What kinds of abilities do they have to check you out, from free accounts or sandboxing or things like that?

    Johnson: Yeah. So, CData.com is our website, and you’ll find all the information that you need there. For any of our installable products, we offer 30-day free trials, and they’re fully functional trials. You know, we’re not rate limiting, we’re not doing anything like that. And then with our trials, you get access to our support team, which we’ve been called world class, we have testimonials on our website if you really wanna dig into those.

    Ashley: Comes from a man who was on the support team, so.

    Johnson: That’s right. [Laughter] 

    Ashley:[Laughter] No, I’m sure it’s really great.

    Johnson: And then, you know, for the Cloud Platform, I think we offer a two-week trial, and so, you can kinda kick the tires on that as well. So, CData.com/drivers is where you wanna go if you’re interested in an installable driver. CData.com/connect is where you can go if you’re interested in that Connect platform, that Cloud Platform for connectivity. And then, you know, you can find us on LinkedIn, you can find us on Twitter, we’re @CDataSoftware. You can find us on Facebook. A lot of our posting there is upcoming events or recent articles or other news announcements, so.

    Ashley: Yeah, and I have checked out your drivers, and you know, an analogy like a Swiss Army knife falls apart very quickly, because you have so many technologies, so many platforms that you interface with—it’s about 50 Swiss Army knives. [Laughter] 

    Johnson: Yeah. So, I’m just gonna plug this—so, I think we’re 250 plus different data sources across 11 technologies. So, somebody out there is doing the math right now in their head to figure out how many SKUs we’re sitting on right now.

    Ashley:  The number of permutations—well, good. It’s really been good talking with you, and I look forward to having you all on again and talking some more, I think, about the low code/no code space and some things that are happening there, so we’ll look forward to that.

    Johnson: Yeah. Alright, yeah. Thanks, Mitch, it’s been my pleasure.

    Ashley: Great. Jerod Johnson, who’s Technology Evangelist with CData.

  • ServiceNow Shifts to Single Unified Agent

    ServiceNow Shifts to Single Unified Agent

    ServiceNow today unveiled an Agent Client Collector (ACC) that makes it possible for IT teams to capture data from software, hardware and cloud infrastructure.

    Jeff Hausman, vice president and general manager for IT operations management at ServiceNow, said this unified agent will enable IT teams to reduce the number of agents they would otherwise have to deploy to collect the data required to monitor IT environments.

    IT teams will also be able to employ incident management tools that can view that data in a way that eliminates the need to install any remote desktop software, Hausman noted. At a time when many employees will be splitting their time between working from home and the office, that approach reduces the level of IT service management (ITSM) friction that many IT teams today regularly encounter, added Hausman.

    ACC surfaces both configuration and performance data in real-time to provide IT teams with greater visibility. That data can then be used to drive a variety of workflows and playbooks that trigger automated incident responses.

    ServiceNow will employ ACC both within its ITSM platforms and IT operations management (ITOM), hardware asset management (HAM), software asset management (SAM), and security operations (SecOps) tools.

    Hausman said ServiceNow will employ ACC alongside agentless tools for managing IT environments as part of an effort to strike a balance between the amount of software that needs to be deployed and the capabilities that agent software enables. Ultimately, ServiceNow will also open up ACC to third-party tools that offer complementary capabilities, he added.

    More than a few IT organizations are suffering today from a phenomenon known as agent fatigue. Each new tool added to the IT arsenal often requires agent software to be installed within a device, application, server or cloud service. The need to reduce the amount of agent software has given rise to agentless approaches to managing IT resources, but the tradeoff is that, in the absence of an agent, the types of data and capabilities that can be enabled tend to be more limited.

    The market research firm International Data Corp. (IDC) is predicting that 75% of Global 2000 IT organizations will adopt automated operations practices by 2023. A major driver of that increased reliance on automation has been, of course, the COVID-19 pandemic. Not only are organizations making greater use of cloud infrastructure, but application users are not often conveniently located behind a firewall in a corporate office. For the last year, IT teams have been tasked with supporting a largely remote workforce. As more of those employees are vaccinated in the weeks ahead, a lot more of them will be returning to the office. However, it’s unlikely most employees will be working in an office full-time even after the COVID-19 pandemic subsides.

    Regardless of how IT is managed, the overall IT environment will continue to become more complex. As such, it’s now more a question of when rather than if organizations will need to revisit how IT gets managed across what has become a highly distributed environment.

  • Fast Object Storage Key to Backup and DR Strategy

    Fast Object Storage Key to Backup and DR Strategy

    The massive shift to remote work necessitated by the COVID-19 pandemic kickstarted many companies’ digital transformation. While this transition had already begun for most organizations, the pandemic sped up the timeline. These events also had an impact on how storage and backup is done. Let’s look at why the right storage solution is key to your backup and disaster recovery strategy, especially when it comes to today’s work-from-home and hybrid scenarios.

    Hybrid Cloud and Storage

    While most companies have moved to the cloud in some form or another, many are relying on a hybrid approach. Enterprises may want to maintain their primary data storage on-premises for performance, security and compliance reasons. The public cloud, though, offers compelling services that can enhance this on-premises data.

    Using both the public cloud and on-premises storage allows for hybrid cloud use cases that are changing the way enterprise IT leaders think about data management and data protection. Enterprises are implementing hybrid cloud architectures to take advantage of cloud archive services, to maintain full copies of their on-premises data for disaster recovery, to protect against ransomware attacks, to take advantage of cloud service bursting and more.

    Businesses retain greater control over their private data by using hybrid cloud services. An organization can store sensitive data on a private cloud or local data center and, at the same time, make use of the robust computational resources of a managed public cloud. A hybrid cloud relies on a single plane of management, unlike a multi-cloud strategy that forces administrators to manage each cloud environment separately.

    Remote Work has Changed the Backup and Recovery Game

    Companies risk losing their most valuable assets – their data – if they lack strong backup and recovery infrastructure. Data loss affects businesses directly, when assets can no longer be leveraged, or indirectly, via loss of customer confidence and regulatory fines and penalties.

    The shift to remote work means that the way storage and backup is done has also changed. If the majority of a workforce is remote now, how do you back up laptops? How do you back up phones? Previously, employees would go to an office and their laptop and other devices would get backed up there – but now, that’s not the case. And with remote work in place for the long haul for many companies, backup methods have to change and adapt.

    Remote work opens up greater risks of information being stolen or lost. If an employee is at home, for instance, and they lose a file on their computer – and they’re used to having their computer backed up by the company – how do they restart that backup? If they have to go through IT, that seemingly simple act can take weeks.

    Self-Service Backup and Recovery

    Organizations need to have more flexibility – a way for individuals to perform backups in a self-service way without having to deal with centralized IT. In this way, people can protect their data themselves.

    With the right storage solution, authentication is built in; storage and backup are designed with security and data protection in mind. Unlike other storage solutions, object storage gives you your own keys and your own passwords, and you only see what you should. It’s very well protected. It’s also very siloed, so that employees can only do what the company wants them to be able to do, and they only have the minimum access they need. So, object storage can help organizations adopt a more fine-grained authentication and security posture; more so than with a typical file storage solution.

    In addition, having this kind of solution in place opens opportunities to provide your employees with the kind of self-service backup and recovery they need in a remote world where speed is of the essence and efficiency is front-and-center.

    Success in a Hybrid World

    The shift to the cloud is in full swing for many organizations, brought on by a variety of business factors and accelerated, in many instances, by the pandemic. It makes sense that the hybrid storage approach would become popular, as organizations naturally want to keep certain kinds of data close to the vest for security and/or compliance reasons. Performance is another consideration when it comes to where data should be stored.

    This gives organizations a lot to think and strategize about – including how to handle backup and recovery in a WFH world. Remote work increases security threats, so organizations need the option to empower employees to protect their own data. Object storage makes this possible with both speed and efficiency. There are many options available today for both cloud and storage configurations, so consider the above points when making decisions that will affect the security, compliance and performance needs of your organization.

  • Applause Delivers Testing Integration Tool

    Applause Delivers Testing Integration Tool

    Applause today made a bi-directional integration capability that simplifies requesting application code tests of, dubbed Applause In-Sprint Testing, generally available.

    Kristin Simonini, vice president of product for Applause, said application owners will be able to, for example, use a Jira project management application to create a ticket that requests a specific piece of code be tested either by a member of dedicated testing team or by a developer that has assumed greater responsibility for application testing within the context of a DevOps workflow.
    Alternatively, that request could be sent to a tester that is a member of uTest, an online community of professional testers that Applause oversees.

    The company also published the results of a global survey of 1,800 quality assurance (QA), product, engineering and DevOps professionals that found 44% of development teams that have shifted application testing left are spending a full workday every week on application testing. More than half the respondents (52%) noted that amount of time spent on testing is impacting developer productivity.

    On the plus side, a full 88% of respondents that release code multiple times per month said their organizations now test features as they are developed versus waiting until the end of application development process, the survey also found.

    The major benefits of shifting testing left cited by respondents are reducing the number of bugs released into production environments (57%), reducing the cost of late-stage bug fixes (48%), and reducing the need for hot-fixes (39%).

    The top two barriers to shifting responsibility for testing left toward developers is they are too busy coding (45%) and that development and quality assurance (QA) efforts remain siloed (37%). Nearly half of developers (44%) said they have issues with context switching as they shift between coding and testing tasks.

    Simonini said Applause In-Sprint Testing will make it easier for application owners to decide when they want to make use of a professional tester versus having a developer run a test. The Applause Product Excellence Platform then enables IT organizations to leverage a software-as-a-service (SaaS) platform to record a series of test case scenarios that can be automatically converted into an automation script without anyone having to write code.

    The overall goal is to simplify testing at a time when the number of applications being built has accelerated in part because of the increased focus on digital business transformation that occurred in the wake of the economic downturn brought on by the COVID-19 pandemic. As organizations realize how dependent they have become on those applications to differentiate themselves, the focus on quality assurance naturally intensifies, noted Simonini.

    There’s no doubt more organizations are testing with greater frequency. The efforts of developers that previously tended to address large numbers of bugs as part of a series of application updates after deployment in production are now being frowned upon. Organizations realize their applications only get one chance to make a first impression with end customers. As a result, the level of attention to detail being applied to application testing has, for better or worse, never been greater.