devopsy.kinsta.cloud

Tag: observability

  • How Low-CodeOps Can Transform DevOps

    How Low-CodeOps Can Transform DevOps

    Over the past year, enterprise adoption of low-code software development platforms surged across the world economy as COVID-19’s economic consequences drove an unprecedented number of digital initiatives. Enterprises are turning to low-code platforms to rapidly build new apps that empower customers, remote workers and external end users conducting business operations or pivoting to new opportunities.

    Accelerating software development, however, only solves half the puzzle of a digital-first approach. Quickly getting solutions into users’ hands requires careful planning to determine how these new applications will function within the organization’s overall software operations strategy. This is often conducted across multiple cloud deployments.

    But the real prize comes when low-code’s underlying principles of automation and abstraction are extended into the DevOps domain of software maintenance and management. The ability to push out a continuous pipeline of software updates with the click of mouse, with streamlined testing and quality assurance, requires a platform that automates myriad, time-consuming tasks associated with building, testing, releasing and maintaining applications in an enterprise-grade production environment. This inevitably leads to the concept of “low-code operations;” or, to coin a phrase, “Low-CodeOps.”

    Indeed, unless the benefits of abstraction and automation — in this case, primarily speed — extend to operations and deployment, the real benefits of low-code development are elusive. A platform that lets you build an app in six days but requires six months to deploy is no solution at all.

    Harnessing the Cloud for Game-Changing Results

    To witness the vital role seamless cloud operations played in business innovation, look no further than biotech giant Moderna and its success in Operation Warp Speed. Moderna used the public cloud to pull together geographically dispersed collaborators, integrate scientific insights and refine its production cycle, enabling the company to deliver its first batch of mRNA vaccine samples a mere 42 days after the COVID-19 virus genome was sequenced. The time-to-value for this solution was definitely warp speed.

    And, it’s safe to assume, expensive. Cloud engineers and DevOps system administrators are highly skilled specialists who roll up their sleeves at 2 a.m. to fix unexpected glitches buried deep in the connected ecosystem of software operations. They figure out how to make the software actually run. Enterprises, no doubt, wish they could hire more of them.

    But the same shortage of skilled IT professionals that fueled today’s demand for low-code software development also impacts DevOps. In this case, the same low-code platform is the solution, empowering a wider cross-functional, collaborative team to reduce operational burdens — in a word, Low-CodeOps.

    Essential Features of Low-CodeOps

    To achieve a frictionless handoff when moving applications and data to the cloud, a production environment leveraging low-code principles will deliver on these key principles of low-code operations:

    Performance monitoring: Daily monitoring and log checks of platform infrastructure are the backbone of modern DevOps operations, ensuring that complex interdependencies are running smoothly. Automated dashboard monitoring can transform hours spent on these repetitive tasks into opportunities for engineering teams to focus on innovation, while still ensuring the health and performance of enterprise software portfolios. The tooling needs to enable automated end-to-end visibility into application workloads and performance characteristics of upstream and downstream services.

    Compliance: Every economic sector is subject to specific government regulations. Software platforms and DevOp teams must keep track of every requirement when integrating new functionality into software upgrades. Banking customers must deploy digital solutions on infrastructure that’s earned ISO/IEC 27001 certification. Healthcare solutions must conform to federally mandated HIPAA requirements. Defense contractors must be FedRAMP certified. Data—the lynchpin of any digital solution—is subject to privacy laws that differ across nations and borders. Imagine having to manually recheck every iteration of your application deployments for compliance. Luckily, low-code operations mean having an automated, system-wide compliance framework in place. This makes short work of service organization (SOC) audits and reporting.

    Security: Cyberattacks that lead to system outages and disruptions are a growing concern in our digitally connected world. Well-resourced cloud giants are not exempt from damaging cyberattacks, such as last year’s SolarWinds hack. Resilience is key. Ongoing security scanning can halt automatic software deployments when suspicious changes are detected. Prevention is always best. Monitoring tools identify vulnerabilities, enabling systems to quickly come back online and help prevent outages that can last for hours or days.

    Infrastructure independence: Widespread enterprise adoption of multi-cloud and hybrid cloud for operational efficiencies has made flexible infrastructure the new standard. Organizations are bypassing single-vendor lock-in to spread workloads across multiple cloud partners. That means DevOps needs streamlined configuration and management tools that can track relationships between resources and keep track of compliance issues. Geopolitical considerations may enter the equation when cloud providers turn out to have close ties with governments antagonistic to other global trading partners. Monitoring instruments that provide observability across the range of vendors consuming enterprise data and applications is mission-critical.

    Resilience: Snafus happen. Platforms can automate continuous domain-level metrics to correlate performance from different sources (including logs, metrics, application tracing and outside penetration testing), speeding up detection and response time. Fast auditing and debugging tools, combined with real-time dashboard monitoring and system-wide alerts, plus an ounce of detection, equals a pound of cure. For agile DevOps teams, reactive firefighting is transformed into resilient firepower.

    Adopting an All-in-One Low-CodeOps Approach

    Low-code’s power to enable frequent iterations of software ideation, creation and deployment may seem contrary to the DevOps mission to minimize risks. However, the same radical abstraction and automation that has made software development faster, higher quality and more accessible to a larger cohort of developers can also reduce the operational burden of landscape management. Every low-code vendor touts the governance features in its platform. But the nuts and bolts of governance require some assembly, including system-wide actionable monitoring, testing alerts and compliance safeguards. Low-CodeOps combines the most efficient, agile, flexible development platform and powerful, streamlined operations tools, infusing both domains with solid guardrails for operational control and predictability. Adopting this approach will greatly increase time-to-value for enterprises, putting innovative digital solutions directly into the hands of end-users with the click of a mouse.

  • Achieving Reliable Observability Part 1 – Making Cloud-Native Observability More Robust

    Achieving Reliable Observability Part 1 – Making Cloud-Native Observability More Robust

    I was having a conversation with a CxO level customer as part of an AIOps/Observability workshop, and from what I could tell, most are confused about how to properly operationalize cloud-native production environments – especially the monitoring/observability portion. Here is how the conversation went.

    “Andy, we are thinking about getting [vendor] to use for our observability solution based on your recent research. What do you think?”

    “Well, I don’t want to endorse any specific vendor, as they are all good at what they do. But let’s talk about what you want to do, and what they can do for you, so you can figure out whether or not they are the right fit for you.” The conversation continued for a while, but the last piece is worthy of being called out specifically.

    “So, we will be running our production microservices in AWS in the ____ region. And we are planning to use this particular observability provider to monitor our Kubernetes clusters.”

    “Couple of items to discuss. First, you realize that this particular provider you are speaking of also runs in the same region of the same cloud provider as yours, right?”

    “We didn’t know that. Is that going to be a problem?”

    “Not particularly. However, you may get into a ‘circular dependency’ situation.”

    “What is that?”

    “Well, as an enterprise architect, I always call for separation of duties as a best practice. For example, having your developer testing the code is a bad idea, having your developer figuring out how to deploy is a bad idea. In much the same way as when your production services run in the same region as your monitoring software – how would you know about a production outage if the cloud region takes a hit, and your observability solution goes down at the same time your production services do?”

    “Should we dump them and go get this other solution instead?”

    “No, I am not saying that. Figure out what you are trying to achieve and have a plan for it. Selection of an observability tool should fit your overall strategy.”

    For those who don’t understand the above conversation, here is the reason why this scenario could be a problem.

    Coming from an enterprise architecture background, we were taught, as a best practice, to operationalize production systems to avoid circular dependencies. This includes not having two services depend on each other, or not to colocate monitoring, governance and compliance systems as part of the production systems themselves. If you were to monitor your production system, you would do it from a separate and isolated sub-system (server, data center rack, sub-net, etc.) to make sure that if your production system goes down, the monitoring system doesn’t go down, too.  The same goes for public cloud regions – although it’s unlikely, individual regions and services do experience outages. If your production infrastructure is running on the same services in the same region as your SaaS monitoring provider, not only won’t you be aware that your production systems are down, but you also won’t have the data to analyze what went wrong. The whole idea behind having a good observability system is to quickly know when things went bad, what went wrong, where the problem is and why it happened so you can quickly fix it. You can check out this blog where I explain this in detail.

    The best practice would be to either:

    1. Consider an observability solution that runs in a different region than your production workloads. Better yet, consider something that runs on a different cloud provider altogether. Although it is exceptionally rare, there have been instances of cloud service outages that cross regions. The chances of both cloud providers going down at the same time would be slim.For example, if the cloud region goes down (a region-wide outage in the cloud is quite possible, and seems to be more frequent of late), then your observability systems will also be down. You wouldn’t even know your production servers are down to switch to your backup systems unless you have a “hot” production backup. Not only will your customers find out about your outage before you do, but you won’t even be able to initiate your playbook, as you are not even aware that your production servers are down.
    2. Consider having your observability solution in a different location/region, yet still close enough to your production services so latency is very low. Most cloud providers operate in close proximity, so it is easy to find one.
    3. Another option is to get a solution that gives you deployment flexibility. For example, there are a couple of observability solutions that allow you to deploy it in any cloud and observe your production systems from anywhere – both in the cloud and on-premises.
    4. You can also consider sending the monitoring data from your instrumentation to two observability solution locations, but that will cost you slightly more. Or, ask what the vendor’s business continuity/disaster recovery plans are. While some think the costs might be much higher, I disagree, for a couple of reasons. First, because monitoring is mainly time-series metric data, so the volume and the cost to transport is not as high as logs or traces. Second, unless your observability provider is VPC peered, the chances are your data will be routed through the internet even though they are hosted in the same cloud provider. Hence, there will not be much more additional cost. Having observability data, all the time, about your production system is very critical during outages.
    5. A very commonly overlooked consideration is monitoring your full-stack observability system. While it is preferable to have the monitoring instance in every region where your production services run, it may not be feasible either because of cost or manageability. On such occasions, monitor the monitoring system. You could do synthetic monitoring by checking the monitoring API endpoints (or do random data inputs and check to ensure it worked) to make sure that your monitoring system is properly watching your production system. Better yet, find a monitoring vendor that will do this on your behalf.

    When/if you get the dreaded 2 a.m. call, what is your plan of action? Just think it through thoroughly before it happens and have a playbook ready, so you won’t have to panic in a crisis.

  • Grafana Labs Aligns with Elastic on Observability

    Grafana Labs Aligns with Elastic on Observability

    Elastic and Grafana Labs have allied to create a plugin through which DevOps teams that are employing Elasticsearch within a DevOps workflow will be able to invoke a monitoring platform, based on Prometheus, that is managed by Grafana Labs.

    Raj Dutt, Grafana Labs CEO, said both companies are dedicating engineering resources to building and maintaining a plug-in for a service, based on open source Prometheus software, that is rapidly becoming a de facto standard for observability in DevOps environments. One of the primary reasons for that expanded usage is that Grafana Labs doesn’t require DevOps teams to move data into a central repository to analyze it, Dutt said.

    The overall goal is to eliminate the “tunnel vision” that results when relying on tools that only visualizes a narrow set of metrics, Dutt added.

    The cloud platform provided by Grafana Labs is based on open source Grafana visualization tools. The primary tool for visualizing Elasticsearch data is based on open source Kibana tools. Grafana, however, is based on a fork of Kibana, so the plug-in represents an effort to bring those two open source communities closer together. The two companies plan to collaborate on including more aggregations, broader query language support and support for space-saving constructs like rollups as the partnership evolves.

    Grafana Labs provides a cloud service that aggregates data, pulled from multiple instances of Prometheus, deployed on clusters and servers. Prometheus, especially, gained a lot of traction as organizations began to deploy Kubernetes clusters that are also based on open source software. Both Kubernetes and Prometheus are being developed under the auspices of the Cloud Native Computing Foundation (CNCF).

    Once DevOps team initially employ Prometheus to monitor Kubernetes, they frequently expand deployments to monitor additional platforms. That shift has resulted in many providers of proprietary monitoring tools to extend the data they consume to include the metrics captured locally by Prometheus. Grafana Labs claims it has more 1,000 customers, including Bloomberg, JP Morgan Chase, eBay, PayPal and Sony, for a total of more 650,000 active installations of Grafana around the globe. Existing customers of Elastic that are paying for Elasticsearch support include Cisco, eBay, Goldman Sachs, Microsoft, The Mayo Clinic, NASA, The New York Times, Wikipedia and Verizon.

    It’s not clear to what degree Grafana and Kibana might ultimately re-converge one day, but it’s clear there will be a fair amount of ongoing cross-pollination across the two projects. In the meantime, DevOps teams should be able to opt for whichever open source visualization framework they prefer alongside Elasticsearch.

    In the longer term, it’s clear greater appreciation for observability platforms that provide more context will drive a wave of consolidation among all the monitoring tools that enterprise IT teams rely on today. As part of the transition, enterprise IT teams might even standardize on a set of visualization tools that would eliminate the need to master different user interfaces. After all, the only way to actually achieve observability is through a set of tools that makes surfacing the context required easier to consume.

  • LogDNA Adds Usage Quota Tools to Help Control Costs

    LogDNA Adds Usage Quota Tools to Help Control Costs

    LogDNA announced today it will enable IT teams to apply usage quotas to help rein in log management costs.

    Tucker Callaway, CEO, LogDNA, said as applications become more complex – thanks, in part, to the rise of microservices – the amount of log data that a DevOps team can collect has significantly increased. IT teams can now assign DevOps teams both hard and soft quotas to limit the amount of log data collected during a specific time period, Callaway said.

    IT teams can set a daily or monthly hard limit on the volume of logs stored or, alternatively, they can also set soft daily or monthly quotas that can apply throttling logic to ensure mission-critical log data will continue flowing through LogDNA as usage volume approaches the limit threshold. It will then be up to each IT team to determine, based on the tasks at hand, whether to enable a DevOps team to exceed those thresholds, Callaway said.

    IT teams can also make use of soft limits to reallocate storage resources from one DevOps teams to another based on how much log data might be generated in a specific time period, Callaway noted.

    LogDNA will notify team members any time a quota is hit, or when an exclusion rule is triggered because a quota has been reached. Those alerts can also be integrated with email systems or the Slack messaging platform. Logs that are not retained are not counted toward the monthly bill, but are displayed and will trigger other relevant alerts.

    In the wake of the economic downturn brought on by the COVID-19 pandemic, IT organizations are, generally, more sensitive to costs. At the same time, many of them are encouraging developers to assume more responsibility for application life cycle management as they embrace DevOps best practices. The usage quotas are intended to enable IT teams to strike a better balance between what can be two competing agendas, Callaway said. IT leaders are not necessarily trying to discourage DevOps teams from collecting log data as much as they are trying to make sure costs are predictable, Callaway added.

    Log data, of course, is one of the core elements of any observability strategy. While observability has always been a core DevOps principle, the number of organizations that have been able to meaningfully achieve it has been limited. However, as organizations start to deploy microservices with multiple dependencies, the need to provide context when troubleshooting those applications has become more acute.

    It’s not clear to what degree more granular control over log data might sway an IT organization toward one platform versus another. However, when it comes time to justify a choice to senior IT and business leaders, it’s always a good idea to address, at some point, how one decision versus another might have a more positive impact on the bottom line.

  • Datadog to Meld Observability and DevSecOps

    Datadog to Meld Observability and DevSecOps

    Datadog is planning to expand the scope of the security and observability services it provides following the acquisitions of Sqreen, an application security platform provider, and Timber Technologies, a provider of a tool, called Vector, for collecting and normalizing log data.

    Ilan Rabinovitch, vice president of product and community for Datadog, said Vector makes it possible to collect, enrich and transform logs and other observability data from both on-premises and in-cloud environments in a way that makes it easier to route that data to wherever it is needed via a data pipeline.

    Sqreen is a provider of a software-as-a-service (SaaS) platform that detects, blocks and responds to application-level attacks using runtime application self-protection (RASP) capabilities enabled by a web application firewall (WAF) embedded within an application.

    Rabinovitch said as IT continues to evolve, it’s clear organizations are looking to embrace DevSecOps best practices in a way that will require security tools that can be embedded within an application, coupled with observability tools that provide more context about the overall IT environment. That context is especially critical, given the amount of data that needs to be analyzed to determine the potential scope of a security threat to an IT environment, Rabinovitch said.

    The Datadog platform is already widely employed to monitor cloud computing environments. There is an industry wide effort to take monitoring to the next level by enabling observability, which applies analytics to provide more context. Surfacing that context, however, requires the ability to normalize data across a pipeline using Vector, Rabinovitch said.

    Of course, the Vector tools are equally applicable to DevOps workflows that have yet to incorporate security technologies. The goal, now, is to make it easier for IT teams, made up of developers, IT operations teams and security specialists, to collaborate by breaking down the various data silos created by IT and security management tools.

    It’s too early to tell what impact the rise of DevSecOps will have on roles within IT organizations. Most organizations will continue to employ a range of specialists that will be able to work more closely with IT generalists, Rabinovitch said. Managing IT is becoming the equivalent of a barn raising, in that it requires members of a community with different skills to come together for a certain amount of time to accomplish a specific task, Rabinovitch added.

    It’s also not clear whether IT organizations view observability as a distinct capability separate from monitoring, or whether they will simply view observability as the next logical extension of existing tools. At the core of that issue is debate over whether observability platforms need to be acquired separately, as part of an effort to replace existing monitoring tools, or whether those tools will evolve over time to provide observability capabilities.

    Regardless of the path forward, it’s apparent the increased complexity of modern IT environments – made up of microservices, containers and serverless computing frameworks – is about to force that issue.

  • SolarWinds Unveils Revamped APM Platform

    SolarWinds Unveils Revamped APM Platform

    SolarWinds is unifying its various application performance management (APM) tools within a single suite accessed via a software-as-a-service (SaaS) platform.

    Denny LeCompte, senior vice president for products at SolarWinds, said the APM Integrated Experience offering combines the company’s AppOptics, Loggly and Pingdom tools to collect metrics, traces and logs along with real user monitoring capabilities and tools to simulate application traffic.

    APM Integrated Experience also provides access to a landing page that identifies what has changed in an IT environment over the previous week and highlights which of those changes might become an issue. As most IT professionals well understand, most IT issues can be traced directly back to a recent change made to an IT environment that was previously running as intended.

    Other capabilities include advanced search tools and the ability to inspect lists and maps for hosts, containers and plugins for web servers and databases. In addition, APM Integrated Experience is integrated with both Amazon Web Services (AWS) and the Microsoft Azure cloud.

    LeCompte said the goal is to make it less expensive for the average IT organization to broadly instrument applications using an agent developed by SolarWinds that can automatically be installed.

    Ultimately, the goal is to enable organizations to better manage their software supply chains, LeCompte added. Of course, SolarWinds is at the heart of the software supply chain breach that involved its Orion network management software. That software runs on-premises. In comparison, APM Integrated Experience is a SaaS platform that is more secure. In fact, LeCompte said, IT professionals that manage software supply chains are generally sympathetic to the challenge the company currently faces. Most of those organizations have dealt with similar software supply chain issues, albeit on a smaller scale, LeCompte added.

    In general, LeCompte said it’s apparent most organizations need access to an APM platform that costs substantially less than rival platforms and still delivers 90% of the capabilities they require. In many cases, IT organizations are paying for extra capabilities that only very high-end enterprises require, LeCompte noted. In fact, one of the primary reasons APM platforms are not as widely employed is most organizations have found rival APM platforms simply too expensive to deploy, said LeCompte.

    LeCompte declined to speculate whether there is a meaningful difference between what has been historically known as monitoring and so-called observability platforms that have emerged in recent months. Regardless of approach, however, it’s clear as distributed IT environments become more diverse and complex, the number of organizations that need more visibility into their application environment is steadily increasing, said LeCompte.

    Of course, with the rise of microservices-based applications, that level of IT complexity is only going to increase. As applications grow and expand, the number of dependencies between microservices starts to increase. Understanding how an issue with one microservice is impacting a distributed computing environment will undoubtedly require more instrumentation. The challenge, going forward, will be determining how best to achieve that goal at a reasonable cost.

  • Application Security and Continuous Observability with DeepFactor

    Application Security and Continuous Observability with DeepFactor

    Today’s modern applications are becoming increasingly complex and released more frequently than ever before, therefore increasing security, privacy and compliance risks. How do you identify the security and compliance risks hidden in your application before you deploy it to production?

    Continuous observability enables DevSecOps teams to observe millions of application telemetry events and find runtime security, compliance and privacy risks within the DevOps pipeline, while providing AppSec teams the ability to set guardrails and have continuous visibility with every build.

    In this TechStrong TV interview with DeepFactor Founder and CEO Kiran Kamity, we discuss the concept of observability and how to apply it for security and compliance. We also talk about the increasingly important role of AppSec and DeepFactor’s new Observability-as-Code API, which enables DevSecOps engineers to leverage observability functionality in their CI pipeline and gate builds.

    The video of the conversation is below, followed by the transcript. Enjoy!

    Transcript

    Mitch Ashley: I have the pleasure of being joined by Kiran Kamity, who is founder and CEO with DeepFactor. Welcome, Kiran.

    Kiran Kamity: Thank you, Mitch.

    Ashley: Great to have you. We’re gonna be talking about observability, super-hot topic, but before we get there, tell us a little bit about yourself, a little bit about your background, and then a little bit about DeepFactor.

    Kamity: Awesome, yeah. I came to the Bay Area about 20 years ago. I did my undergrad at the Indian Institute of Technology in India in Chennai, and then I came here to Stanford for my grad program. Once I came to the Bay Area, you get addicted to the entrepreneurial spirit, you never leave, this becomes your home. Happened to most people—happened to me as well.

    So, the Bay Area is my home, and I’ve had a mix of enterprise and infrastructure cloud computing, virtualization, and DevOps/AppSec kind of background over the last 20 years. I’ve done two companies as a Founder, which were acquired by other companies like Citrix and Cisco. My first company, RingCube Technologies, was acquired by Citrix. We built a container run time for the Windows Operating System before the Docker days. This was acquired by Citrix in 2011, so I was responsible for some of the product teams at Citrix after the acquisition.

    Then the bug bit me again, and I started ContainerX, which was a turnkey containers in a box type company, which was acquired by Cisco, and it’s now called Cisco Container Platform. While at Cisco, I was Head of Product for the Cisco Cloud Business Unit, which included container—container networking, OpenStack, and some of the other initiatives that they had.

    During that time, as I was looking at what to do next and analyzing the market, a lot of the learnings came to the following. There’s a lot of momentum that has happened so far in cloud native architectures, DevOps, et cetera. And the learnings were that the next wave of innovation is going to be as DevOps becomes DevSecOps. And that’s why I started DeepFactor, and we believe that the next wave of innovation in the world of DevSecOps is observability.

    Ashley: That’s something I both relate to and also am just fascinated by other entrepreneurs is that story of, you’re working on what you’re doing now and you’re still kinda looking at, “Well, what’s the next thing coming up, and what might I go do next?” and then you go off and solve the next problem.

    Kamity: Yeah, absolutely.

    Ashley: Start another company to do that—like you said, you get the bug, it’s kind of a fever. [Laughter] It definitely is.

    Well, cool. We’re talking about observability, and it does seem to be the hot topic of kinda late 2020, early 2021. So, let’s first start with kinda how did we get there—get here? How did we get to this place where suddenly observability is a thing and not only a thing, but important to a lot of people?

    Kamity: Yeah. Just to qualify it a little bit, our focus at DeepFactor is applying the concepts of observability for security and compliance, which is truly novel, and there’s very few companies that are trying to do that.

    Ashley: True. Observability can mean just sort of the next elevated monitoring, right?

    Kamity: Exactly.

    Ashley: So, you’re [Crosstalk] around security.

    Kamity: So far, a lot of the observability plays have been in the logs metrics tracing. Applying the concepts of observability, truly as the next generation of evolution of APM. But at DeepFactor, we’re leveraging the concepts of observability for identifying security and compliance risks in your applications before they go to production.

    So, to answer your question about how did we get here, let’s take a step back and analyze what has happened over the last few years, with respect to application development, right? Apps have become a lot more complex than they have ever been today. I mean, think about it—a common application is made of a bunch of different languages. You have a Java service, you have a Node.js service, and you have Angular in the front end, you have lots of different pieces, you have a database, et cetera.

    You have a lot of third parties—third party libraries that are baked into your services, third party components that are not built by you such as Nginx and MySQL and MongoDB, et cetera, that become part of your application.

    Ashley: A lot of open source, right, in that stack.

    Kamity: A lot of open source. Applications are now also being deployed in a bunch of different ways. You have applications that are not containerized, still mainstream. You have containerized applications, you have full on Kubernetes applications, and you have land applications as well, so the deployment of applications has become complex as well.

    So, in this world of applications becoming a lot more complex than they’ve been, what essentially happened from a security perspective is that the attack surface area of your application has significantly increased. It is the highest it’s ever been so are, and it’s only gonna go that way.

    Let’s take a look at a few other trends that have bene happening in industry. The velocity of releases—that has gone up as well due to DevOps. You’re no longer building or releasing an application once at three months, once at six months. You’re releasing that multiple times a week with DevOps, and different companies are different aspects of that spectrum, there.

    Now, when things are more complex and they’re being released at such a rapid pace, there is no way humanly possible for your AppSec teams to find all of the security risks and compliance risks that lie in your application before they deploy to production. At the same time, the risk of these issues being in production is higher than it’s ever been. If you think about it, the sophistication of the bad actors has increased significantly as well over the last few years, and therefore, the nature of the breaches, the size of the breaches, and the damage that they can cause has increased as well. Coupled with that, privacy concerns have increased, resulting in standards like GDPR and CCPA and all of those things emerging as well.

    Now, in such a landscape, you’ve got to automate identifying the security and compliance risks that are hidden in your application before you deploy your application to production. And that is where the current tools of just code scanning or identifying issues based on signatures or just DAST scanning your application, et cetera, don’t cut it. You need something that watches your application while the application is actually running in both pre-production environments as well as in a canary deployment of minimal production deployment before you release it to your broader production deployment.

    And in that environment, you’ve got to be able to watch the millions of events that are happening inside your application, take that data, analyze and find the needles in the haystack, and therefore determine where the security and compliance risks are and pass on all of this information with rich evidence such as stack traces, metrics, and whatever it takes to fix these issues back to the developers. And a lot of this has to happen in a seamless manner that is nicely tied into your DevOps kinda pipeline, and that’s where observability applied to security and compliance comes in, and that’s exactly what we call continuous observability at DeepFactor.

    Ashley: Interesting. I know, talking with you previously and kinda hearing about DeepFactor, you all talk about start left, not shift left—this idea that, really, it starts from the beginning, not, “Let’s kinda push it as far left as we can.”

    Kamity: Yeah, absolutely. If you think about it, we’ve had operator specific tools emerged in the last five years with respect to security. So, you’ve had container security tools that identify issues in your Kubernetes configuration or your Docker configuration or vulnerability scanners that identify issues in your host operating system if you’re deploying your app in a VM.

    Now, those are all the types of insights. Are there any misconfigurations that could lead to security issues that are very relevant for the Ops people? And those tools are coming up with the idea of shifting left, because you need to find these configuration issues before you go to production so that the Ops people can deploy these applications properly.

    But when it comes to developers, their agenda is not securely deploy your application. They don’t care, honestly, about how you’re Docker or Kubernetes is configured, for the most part. They care about what is in their application, securing at the source. And when the agenda for Dev teams is to secure at the source, then all you care about is—what is my application doing, whether it’s my application or some third party that I brought in, or just my interpreter itself when it is running? And can I identify the risks involved and understand my attack surface within the application ahead of time? And that’s why we call it start left.

    So, we think that operators care about shifting left. Developers should truly care about starting left.

    Ashley: Yeah, it’s interesting, because your point about code scanning and DAST approaches to security is, from a code standpoint, you do wanna be writing a secure code, so it’s a valid thing, of course, to be doing. But you mentioned Lambda, you mentioned, of course, infrastructure as code, essentially, where all this is dynamic, right, in our environments today, and more so, not less, every day, right?

    Kamity: Yes.

    Ashley: Every day, it gets to be more dynamic. Now, you’re really talking about the whole context of the environment with the application. Kind of, if you thought about it in the network, you’d say all seven layers, right? If you think about it in the security stack of software, it’s application plus all of the other things, right?

    Kamity: Yes.

    Ashley: It’s interesting what—how do you find people are accepting this idea of run time information being provided with context, you know, this rich information you were talking about, is that something new to developers or like, “Oh, my gosh, I’ve been waiting for this forever” or, “Well, this is really cool. Look at—let me see what I can do with it”? What’s more the experience that developers have?

    Kamity: Yeah, so, I think different organizations react differently to something like this. The old way of thinking is—oh, developers don’t care about security or compliance.

    Ashley: I’ve never met a develop that wants to write insecure code or unsecure code. [Laughter]

    Kamity: Exactly, right? That is exactly what’s happening now where, given the consequences of shipping insecure applications and given the need to be more and more secure at the source, I think developers—and just simply because it’s just not humanly possible for your AppSec teams to identify all these things. Given the talent shortage of the AppSec skill set as well, it is, the responsibility if identifying these risks has to be borne by the developers. Because at the end of the day, the developers are the ones that are writing the code and you’ve got to make sure that you’ve shipped secure code.

    Engineering leadership and AppSec teams in various organizations that we’ve been working with have been fully accepted. In fact, they’re fully aware. In fact, a lot of them want this kind of concept, it’s just the lack of tooling today. Because if you think about the current tools that are available, they’re doing the best they can with the existing set of tools.

    I mean, they’re scanning the code, they’re trying to scan the application from time to time—but those tools don’t lend themselves well, nicely into seamless integration into your CI pipeline, and therefore, giving developers the insight about, “Hey, here’s where we saw your application did this.” That can only happen with observability tools like what we’re building, because we watch every thread of every process of every service in your application when it is running in test and even canary deployment in your part of production before you deploy that thing widely to a larger section of production. And that is the kind of insight developers do care about.

    What we have gotten repeated feedback as we have been building this product with a few development partner type enterprises so far is that, the insights that we have needs to be enriched with richer evidence information so that developers can actually take that evidence information and directly go fix the problem. So, we’ve augmented our insights with stack traces wherever appropriate. You know, obviously, the good old things like CVE numbers and things like that, we have that.

    But what we focus more on is not just giving you a list of CVEs, but complementing the insights that your software composition tools give you by giving you run time usage information of that CVE. For example, we can tell you if a tool like Snyk will tell you there are 250 vulnerabilities in your application, DeepFactor will tell you, “Hey, there are 250 vulnerabilities according to what Snyk is telling you. Fifteen of those things have been used by your application at this time, when your application was executing here, ad here is the function, here is the call stack.”

    So, that will make it easy for a developer to go, “Okay, now I don’t have to get overwhelmed with fixing all of the 250 things or adding that to my backlog, I can actually prioritize which ones to fix first, because that’s what my application is actually doing.” It’s not about the whole thing that has been checked into your source code, it is what your application’s actually doing that will help you catch the more important things. It’ll help you also reduce the number of alerts that can inundate you.

    Now, with all of these changes, the rich evidence, reducing the number of alerts, and to the point, alerts based on what your app is actually doing, developers are finding it more and more meaningful, our insights. And as we grow the company, that’s gonna be our primary focus.

    Ashley: That’s always been the challenge with vulnerability scanners and code scanners is, yes, I get all of those results. What of those are real? Yes, they’re all theoretically possible. I may not be using it in a way that actually exposes me to that CVE, that vulnerability.

    But what you’re saying is, you’ve actually detected the use in the application or the software stack of something around that CVE that’s worth looking at. Hopefully, you’re cutting down on the number of things that are actually, they go look at. You can ignore the rest, if you will, or at least know that they’re much less likely to be an issue.

    Kamity: Yeah. We do not give you too many alerts. We give you the alerts that are relevant for your application because we watch the millions of events that are happening in your application in every part of your application, across various layers of the stack—like, system caller interface, library calls that your application has made, network behavior, web and API, all of that.

    And based on that data, we treat it as a data problem. So, we take that data and then we try to identify anomalies and detect that these are the risks that lie in your application from a security perspective, and this is how it maps into compliance. So, we will tell you that these are the security risks in your application and in our next module of the product, we’re gonna tell you that this can hurt your PCIDSS, and here’s the number, basically.

    Ashley: Mm-hmm. It definitely is gonna improve your relationship with the security and compliance team, right? If you’re solving these problems before it gets to them, they have to ask, “Well, did you do anything about this? Are we susceptible?”

    Kamity: Yeah.

    Ashley: Talk a little bit about—so, where in the tool chain and the workflow pipeline, where do you fit? Where does DeepFactor play?

    Kamity: Yeah. The best way to think of us is, if you take a step back, if you are thinking about your DevSecOps initiative in total, you want to plug in into a bunch of different points from a security perspective into your CI pipeline. Your developers write code. The next step should be you try to find risks in your code. That’s where the SaaS tools come in, they find the risks that are there in your code.

    Then you create your build artifacts—let’s say, your executables, your container images, and whatnot. Then you try to find risks that lie in your build artifacts. That includes your code, any third parties that you’ve compiled together and created your services. It’s still static, though. You’re not running them yet. Tools that allow you to identify risks in your build artifacts could be container scanning tools, like image scanning tools that scan your container images and find issues or software composition analysis tools like Snyk and Black Dock and tools like that. These tools find vulnerabilities that exist in your components after they’ve been built.

    Then you run your application, you’re testing your application. Either you’re testing it in a test environment or you’ve deployed and you’re testing it in a minimal configuration of your—like, a canary deployment in your production. During that time, you’ve flicked DeepFactor on. DeepFactor doesn’t require any changes to the code of your application; you’re simply running your application.

    And that’s where we’ve built a significant amount of IP. You know, my Co-founder, Mike Larkin, he’s the CTO, he’s the father of the OpenBSD hypervisor. And our senior engineering leadership members came from Qualys and FireEye, having been in the area for 20 plus years working on security products, each.

    So, we’ve created something that involves zero touch, pretty much. You don’t have to touch your application at all, you simply run a command called DFCTL, just like KubeCTL, we came up with DFCTL. So, you do DFCTL, run your app; or DCFTL, run your container; or you drop it into Kubernetes. We have a webhook admission controller that you simply drop into Kubernetes, and then any pod that you want to be monitored, you simply can specify a config file and that pod gets monitored automatically with DeepFactor.

    Now, the way DeepFactor inserts itself is completely behind the scenes. It’s language agnostic. You don’t have to worry about, is it Java, is it Node.js—DeepFactor figures out that it’s Java or Node.js and does some specific things and finds out more information.

    So, you simply run this, your application is running in your test environment or your minimal production environment, DeepFactor observes everything that it’s doing. When the tests are done, it tells you—you throw your automated tests at it while your application is running, just like normal. And then when it’s done, DeepFactor gives you all of that information about what insights have been found and what’s the evidence information that’s pointing to the insight. Is the issue or the risk presenting the attack factor lying in the code that you wrote? Is it lying in a third party that you brought in, or is it at the interpreter level itself? All of them are risks as far as the attacker is concerned that could be exploited, so you need to be aware of where your risks are.

    Ashley: Now, are developers putting in some library calls or DeepFactor code into their application stack as well, then? Is that how you’re getting into the software, knowing what’s happening inside as well as outside?

    Kamity: Actually, that’s the beauty of our IP, and we’ve filed a few patents in that area. Developers do not need to write any code to change, or change anything in their application. You simply enter DFCTL and run your app and DeepFactor inserts itself into the application at your run time, completely seamless to the developer, and when it’s done, it unplugs itself and your application and gives you all the insights.

    Ashley: Wow. That sounds pretty cool. I can imagine that’s a big plus for developers. [Laughter] Yet something ls they don’t have to debug, right?

    Kamity: Exactly, yeah. That’s the significant part of our IP.

    Ashley: You know, it’s interesting you mentioned your team comes from a number of other security companies that have been around for a while—you know, you mentioned Qualys and others starting in the scanning, the vulnerability management space some time ago; that’s about when I got into security, too.

    What do you think we’ve learned from those types of companies and those experiences that we’ve sort of changed our perspective about where we are today, just given—yes, we have much more complexity in software, much more porous security issues, but what have we learned from all of that security scanning that’s helping us?

    Kamity: Yeah, no, I think there have been numerous, numerous learnings, and the learnings have evolved over time as the landscape of application development has changed as well. You know, you’ve had tools in the past that were designed based on the applications of the past. But tools of today and tools of tomorrow are different, because the applications of today or the applications of tomorrow are definitely trending in a different direction.

    One example of the thought process itself is that, you know, when things are in single digits or a lesser number, it’s okay to have manual scanning, manual testing, once in a while test it and then find information, which is kinda the old kind of scan once in a while, identify it, or bring in a pen tester once in three months, you know, check some boxes and get it done kind of thing.

    Those are all great and they’re still relevant even now. What has become even more important is why you do those once in a while scans and checks. You’ve got to make sure that you get in the habit of securing at the source. You can no longer imagine that your infrastructure or the perimeter is gonna secure your application properly. You’ve got to make sure that you build your application securely to start. And that fundamentally has changed the kind of tools that we need to build in order to equip the developers who build applications to secure at the source.

    At the same time, I think the role of AppSec has become even more important than it has ever been. Because, you know, sometimes I keep getting asked, “Oh, are you saying that developers should use these security tools? That means the AppSec teams are no longer relevant?” That is totally wrong, because AppSec teams, their job is increasingly more relevant. Because if you take a step back and you look at the number of attack vectors that are there in your application, we just talked about how your pipeline should be architected—secure at the source, making sure that you scan your code, scan your builds, test your running applications, and identify risks in all of these areas—who ties them together? Who comes up with this overall strategy? Who comes up with that triaging process and passing the evidence back to the developers, identifying which ones are important and which ones are not?

    All of that should come from the AppSec person in the team—AppSec or if AppSec is assimilated into your DevOps and we call it the DevSecOps person, that’s the role of this person is to make sure that he or she is the shepherd of your CI pipeline from a continuous security perspective and then feeds all of that information back to the relevant parties. Some of that could be your developers, some of that could be to your engineering leadership, and some of that could be to your security leadership.

    Ashley: Mm-hmm.

    Kamity: That is how I see the organizations of tomorrow shaping up, learning from what has happened over the last 15 years to 20 years of innovation in this area.

    Ashley: I think that makes a lot of sense. Otherwise, (a) we’re just throwing tools at it, everybody’s doing whatever and who knows whether that’s helping us improve and improve security. And you mentioned the data coming out of it, right? That can also be used for continuous improvement, continuous learning, what are trends that are happening in our application?

    Suddenly we’re seeing much more of this technique or this architecture or this environment being deployed that, you know, not always do the security people have exposure to that early on in the process. So, it’s a good way just to know what the heck is goin’ on and then also, you have an idea about how all of your security tools, developer or non-developer, fit into an architecture that makes sense.

    Kamity: Yeah, exactly. I mean, imagine flipping a switch and identifying, “okay, these are all the touch points that my application has and these are the millions of things that it’s doing” and condensed very well to, “these are the red dots in that whole story, and this is where your risks lie.” That’s powerful, and I think it’s the AppSec person’s job to set all of that up and make sure that a framework is created and then say, any time something that comes up, you know, make sur that you pass it on to the developer with the appropriate evidence and work with the engineering team to decide.

    Because when you have so many different services and so many different languages, the AppSec people, especially because they’re already short staffed, cannot possibly know everything about the business logic of the application, resulting in those decisions that could have been made to bring in component X and component Y and whatnot.

    Ashley: Mm-hmm. You know, Kiran, one of the things that I’ve said over time is that security people need to get a greater understanding of software architecture. They don’t need to have to be developers, right, that’s not what their skill set is, nor necessary their desire. But you can’t treat security as this hard outer shell, you know, we just keep making it smaller and smaller. And now all you’ve got is thousands or hundreds of thousands of small outer shells and now you’re not securing anything, any more.

    It seems like the AppSec team can play a really important part of helping bridge that gap of what are we doing specifically, what are the architectures that we’re using, what might be some of the security risks that maybe a security engineer is just not fully up to speed on or trying to keep up to speed on as it’s changing?

    Kamity: Yeah, absolutely. Imagine the power of flipping the observability switch and the AppSec person going and saying, “Hey, Dev team, we noticed that in this particular piece of your code, you’re using an API that has been deprecated by the operating system” or, “In this particular piece of code, you’re invoking memory improperly, because this tool is showing me exactly what was done when this particular area of memory was allocated with the wrong permissions. Or, “This is an area of code where your interpreter is doing something wrong. Is it possible for us to go change the version of the interpreter that we’re using, maybe upgrade it to the latest version or put a patch?” Or, “Do we need to, in order to mitigate this thing, assuming that there’s no fix in the interpreter, do we need to go put something in our WA in our infrastructure to block the particular attack from happening?” Or, “Should we report this problem upstream to Java itself, the Java community itself, so they can go fix the problem in there?”

    So, lots of different things that could be done once the AppSec team is empowered with this much rich information about what’s happening in the application.

    Ashley: You know, something I don’t know that we talked enough about, we talked about how rapidly and how quickly we’re delivering software applications now. The environment of the software, the API calls we’re using, the libraries, the services, interpreters—all of that is changing in parallel very quickly. Sometimes you don’t feel like you have time to keep up with it, or that’s all you do is spend your time keeping up with that, not kinda delivering business value.

    I think folks that are maybe not involved too deeply in the software creation process really realize not only the complexity but how much change there is to deal with. There’s a lot that goes into that. We talk about technical debt or backlog or whatever—it’s a lot more than that, there’s much more to it. [Laughter]

    Kamity: Absolutely, there’s a lot more than that, and I think having an informed security team as well as, you know, coupled with a structure and the right tools baked into your pipeline, which results in informed Dev team, can result in goodness overall. You’ll end up identifying security and compliance risks earlier in your Dev cycle. You’ll end up fixing those things earlier, and therefore, you will end up shipping a much more secure application, and it’s a cycle. Every time more developers check in code, you know, this kind of identification happens automatically, and you end up identifying more and more risks and fixing more and more things, resulting in more and more secure applications into prod.

    Ashley: Mm-hmm. Awesome. Well, what are some things folks might have access to you can help them get started? Obviously, they can download HIV testing product or get involved that way. Do you have some other tools or things that they might be able to use to make it approachable to get into this kind of an approach?

    Kamity: Yeah, they can go to DeepFactor.io to check out the product, understand more about the product. And if they want to get started, they can either download the community edition that is now free for 10 user teams or less. If you need any help with setting up the product, you can reach out to our Customer Success person.

    We’re also giving away a free DevSecOps Risk Assessment to companies that are working with us as we expand our organization more. This will help any engineering team or any security team to get a free assessment of what DeepFactor can find in their applications so that they will be able to identify, “Okay, these are the kinds of things DeepFactor finds, and it’s useful for me and here’s where I need to go next” kind of thing.

    So, that DevSecOps Risk Assessment is completely free. Contact us, hello@deepfactor.io or just go to the website to learn more.

    Ashley: Awesome, great. I know you’re also speaking at the Cybersecurity 2021, I think you have a couple of sessions there, right? The dates are March 4th and 5th for that event.

    Kamity: Yeah. We have a couple of speaking sessions at Cybersecurity ’21. We have a virtual booth there as well, so do check us out. There is a—you know, our team can answer any questions or even try to explain to you or help you understand what DeepFactor can do and identify in your application.

    Also, today, we’re launching Observability-as-Code. It’s a new module that we’ve created, and a product that allows people to bake in observability into their CI pipeline, similar to how Infrastructure-as-Code enables DevOps and DevSecOps folks to spin up, spin down infrastructure in their CI pipeline itself and other related scripts. You can do, now, the same thing with observability for security and compliance with DeepFactor. So, you can use our API, we have a Swagger doc ready, you can use our API to invoke DeepFactor as part of your CI pipeline, get the results, triage the results, use that results to block, if there’s any P1s you want to block the _____, you can do that. You can do things like that with the Observability-as-Code API that we just announced.

    Ashley: Excellent. Well, I hope folks will check you out. I think it’s a great path to be going down, and I’m sure developers appreciate writing more secure code, but completing an application that’s more secure that they’re involved in developing.

    So, Kiran Kamity, great to have you here, CEO and founder, co-founder—I guess founder [Laughter] of DeepFactor.io. And be sure and check out the speakers that you all have at Cybersecurity 2021. It’s a great conference and we look forward to hearing more from you, there. Thanks for joining me today. Great to talk with you.

    Kamity: Thank you much. Appreciate the opportunity.

    Ashley: You bet.

    Kamity: Bye.

  • AppDynamics Survey Reveals High IT Stress

    AppDynamics Survey Reveals High IT Stress

    A global survey published today by AppDynamics, a unit of Cisco, suggests the last year took a substantial toll on IT teams at a time when many organizations either launched or accelerated digital business transformation initiatives.

    The survey, of 1,050 IT professionals in organizations with revenue of more than $500 million, finds organizations, on average, implemented digital business transformation initiatives three times faster in 2020 than they did in 2019 in response to the economic downturn brought on by the COVID-19 pandemic. That increased level of activity, however, appears to have come at a significant cost. The survey found 89% of respondents felt they are under immense pressure at work, with 84% admitting to having difficulty switching off after work.

    The survey also found stress levels are, not surprisingly, rising as well, with 81% of respondents experiencing increased frustration and 63% conflicting more with colleagues.

    The root cause of these issues cited most often include new sets of priorities and challenges (80%), technology sprawl and a patchwork of legacy and cloud technologies (78%), acceleration to cloud computing (77%) and multiple, disconnected monitoring solutions (74%).

    A full 85% also identified cutting through noise generated by increasing volumes of data to identify root causes of performance issues will represent a significant challenge in the year ahead. A total of 92% also noted the ability to link technology performance to business outcomes will be what’s really important to delivering on innovation goals over the next year.

    Three-quarters (75%) of respondents said they are already struggling to manage overwhelming “data noise.” Nearly three-quarters (73%) also noted they fear the inability to link IT performance with business performance will be detrimental to their business in 2021. Three-quarters (75%) of respondents also noted their organization needs to connect full-stack observability to business outcomes within 12 months in order to remain competitive. Two-thirds of respondents (66%), however, said right now they do not have the resources and support they need.

    AppDynamics regional CTO Gregg Ostrowski said the survey makes it clear that existing approaches to merely monitoring IT platforms are not going to be sufficient, at a time when observability across the full stack of IT platforms being employed is now absolutely essential. Rather than convening war rooms where IT teams responsible for different platforms compare reports in the hopes of identifying the root cause of an issue, Ostrowski said modern observability platforms provide a holistic view of IT environments that IT teams need to proactively resolve issues before they impact the business.

    That level of visibility will also play a crucial role in reducing the stress levels of IT teams that, in many cases, are now in danger of suffering from burnout, Ostrowski added. Of course, the more fatigued an IT team is, the more likely it is mistakes will be made.

    It’s not clear to what degree IT teams are embracing observability as a core IT management concept. The AppDynamics survey noted that 71% of respondents are wary of misleading claims from vendors who are re-labelling monitoring tools as observability platforms. Regardless of how observability is achieved, the general consensus remains: whatever observability means within the context of IT, most organizations lack it.

  • Grafana Labs Adds On-Premises Observability Platform

    Grafana Labs Adds On-Premises Observability Platform

    Grafana Labs announced today the availability of a unified Grafana Enterprise Stack that enables IT teams to deploy a curated observability platform, based on open source tools such as Prometheus, in their own on-premises IT environment.

    Tom Wilkie, vice president of product for Grafana Labs, said Grafana Enterprise Stack, in addition to providing a version of Prometheus optimized for distributed computing environments, includes curated instances of open source Grafana dashboards, Loki log aggregation tools and the Cortex query engine.

    Grafana Enterprise Stack also includes plugins for visualizing data from other commercial tools in Grafana, such as Splunk, New Relic, MongoDB, ServiceNow, Oracle and Snowflake. In addition, it includes reporting tools and enhanced authentication and security controls.

    Grafana Labs

    Developed under the auspices of the Cloud Native Computing Foundation (CNCF), Prometheus has gained traction as a monitoring platform used primarily in Kubernetes environments. However, Wilkie said it’s only a matter of time before IT organizations expand their use of Prometheus to include a wider range of platforms. The Grafana Enterprise Stack facilitates that transition by making it easier for organizations to deploy a complete observability platform based on open source software, said Wilkie.

    While open source software has a key role to play in enabling observability, most IT organizations today rely on a range of proprietary tools to monitor IT environments. Many of those tools will be replaced, in the months ahead, by observability platforms that provide more context than individual monitoring tools currently do. However, it’s not clear yet if IT teams will prefer open source or proprietary platforms that, in most cases, incorporate open source components.

    As IT environments become more complex, the tools most organizations rely on today don’t surface enough actionable insights. Most IT teams today still waste a fair amount of time navigating amongst multiple tools in an effort to discover the root cause of a problem. It’s not uncommon for IT teams to take days to discover the source of a problem that is then fixed in a few minutes. The expectation is that observability platforms will sharply reduce the amount of time and effort required to investigate, for example, a performance issue.

    Of course, to achieve that goal, applications themselves will need to become more instrumented. A mix of open source and proprietary agent software will make it easier to collect traces, logs and other metrics at a reduced cost. In many cases, IT teams have been reluctant to fully instrument an IT environment, because of both the cost of proprietary agent software and the inherent challenges associated with deploying and managing it across an extended enterprise. However, as the deployment process for agent software becomes more automated, organizations are finding it easier to instrument their applications.

    It may be awhile before IT environments are fully instrumented. However, as it becomes easier to collect traces, logs and metrics, the level of observability that can be achieved across an enterprise will undoubtedly increase. The challenge and the opportunity then becomes employing the greater visibility provided by these platforms to manage the sprawl of applications and platforms across IT more proactively.

  • Dynatrace Extends Reach of Observability Platform

    Dynatrace Extends Reach of Observability Platform

    Dynatrace this week updated its observability and software intelligence platform as part of an effort to expand its reach across modern enterprise IT environments.

    Announced at the online Dynatrace Perform 2021 conference, the company’s namesake platform can now capture logs from Kubernetes and Red Hat OpenShift as well as Amazon Web Services, Google Cloud Platform and Microsoft Azure cloud platforms.

    In addition, Dynatrace is adding support for open source log data frameworks such as Fluentd and Logstash. Those logs can now be centrally viewed via a Dynatrace Log Viewer that enables IT teams to search, analyze and segment both real-time and historical log data.

    Dynatrace also added a Cloud Automation Module based on Keptn, an open source tool for automating configuration of observability tools, orchestrating code tests and quality checks. Keptn was created by Dynatrace and is being advanced under the auspices of the Cloud Native Computing Foundation (CNCF).

    The company also launched a Software Intelligence Hub to make it easier to access more than 500 pre-configured integrations, as well as access tools for creating integrations without having to write code.

    Dynatrace also announced it has extended a previously launched Application Security Module to now include support for Node.js applications. In addition, the module includes the ability to enforce security policies against applications instrumented with its agent software, and offers extended capabilities to enable the detection of Kubernetes cluster vulnerabilities, as well.

    Finally, Dynatrace has added a Session Replay capability to its Digital Experience Module to enable IT teams to capture a mobile application experience.

    Steve Tack, senior vice president of product management at Dynatrace, said the company is moving quickly to add support for platforms and technologies widely employed within cloud-native applications to provide IT teams with a single platform through which they can manage monolithic and microservices-based applications.

    With the rise of microservices, IT environments are becoming too complex to manage without some type of observability platform. Historically, cost concerns limited IT teams to employing application performance management (APM) platforms, such as Dynatrace, only for their most mission-critical, monolithic applications. It’s now become apparent that observability platforms capable of correlating log and telemetry data captured from applications and infrastructure are required to effectively manage a modern IT environment.

    As part of that effort, Dynatrace is now collecting data from additional sources alongside the agent software it provides. That data is then fed into an observability platform, that Dynatrace has augmented with a Davis AI engine, to identify anomalies and other potential issues by employing machine learning algorithms.

    It’s not clear to what degree IT teams completely trust the recommendations surfaced by any AI engine. However, as platforms such as Dynatrace collect more data from an IT environment, the more accurate AI models tend to become. In fact,  IT teams that don’t have access to AI capabilities might soon be at a distinct disadvantage, compared to other IT teams that do.

    In the meantime, the battle for observability supremacy continues to intensify, as incumbent providers of APM platforms such as Dynatrace look to forestall a host of startup rivals, all promising greater visibility into IT environments at a lower cost. The challenge now is determining what level of visibility is achieved, and at what total cost.