Backslash Security today added support for C, C++, Ruby, Rust and Scala along with integrations with on-premises instances of DevOps platforms from GitHub and GitLab to its application security platform.
In addition, DevSecOps teams can now discover “phantom packages” not defined or controlled by the original application developer. The Backslash Security platform detects these phantom packages in open-source software code, even if they are not declared in manifest files, to identify vulnerable transitive packages.
A revamped user interface makes it simpler to understand which vulnerabilities are in use and therefore exploitable within their codebase.
DevSecOps teams can create workflows that automatically generate tickets and notifications for Jira, Monday.com, ServiceNow, Slack and Microsoft Teams.
Finally, Backslash is adding support for role-based access controls to make its platform more secure.
Amit Bismut, head of product management for Backslash Security said support for these programming languages will extend the reach of a platform designed to replace legacy static application security tools (SAST) and software composition analysis (SCA) tools with a single integrated application security platform that manages secrets, generates software bill of materials (SBOMs) and is integrated with the Vulnerability Exploitability Exchange (VEX).
The overall goal is to eliminate the number of false positives generated that increase the level of cognitive load on developers by providing DevSecOps teams with an application security platform that makes it simpler to identify and prioritize remediation efforts, he added. That approach eliminates the flood of alerts that developers are inundated with when organizations, instead of shifting responsibility left toward developers, simply “load left,” said Bismut.
Backslash Security previously supported Golang, Java, JavaScript and Python in addition to providing integrations with cloud services from GitHub, GitLab and Microsoft.
Adding support for additional programming languages and on-premises platforms will make it simpler for DevSecOps teams to consistently manage application security across hybrid IT environments made of different classes of applications that all have multiple dependencies, said Bismut.
Most application developers today still have little to no cybersecurity expertise. Many organizations are attempting to shift responsibility for application security left toward developers, but without the right tools at their disposal, it’s not likely they will succeed. It’s still fairly common for a relatively simple SQL injection attack to be successfully used to compromise an application environment, noted Bismut.
The Backslash platform makes it simpler for developers to understand how workflows in their code could be exploited, instead of relying on cybersecurity professionals to discover vulnerabilities that might not be present in a production environment, he added.
There’s no doubt cybercriminals will continue to focus on exploiting weaknesses in software supply chains. The attacks may not be overly sophisticated, but given the current odds of success, the targets are too tempting to ignore. The issue organizations must come to terms with is the increasing amount of liability those security flaws represent, as governments around the world start to hold organizations more responsible for the security of the applications they deploy.
Harness today revealed it has acquired Split Software as part of an effort to extend the features management capabilities of its DevOps portfolio.
As with all Harness add-on tools, the features management platform developed by Split Software will continue to be made available across multiple continuous integration/continuous delivery (CI/CD) platforms even after Harness has completed its integration.
Harish Doddala, senior vice president and general manager for Harness, said Split Software adds an experimentation capability that goes beyond the existing features management capabilities that are currently provided to enable DevOps teams to better manage multiple software development projects.
Split Software president Trevor Stuart said those experimentation capabilities were created to make it simpler for organizations to progressively provide a range of capabilities to different types of end users after an application has already been deployed in a production environment.
Using Features Management in Production Environments
Features management is already widely employed in A/B testing but the number of organizations using feature management to provide varying levels of application experiences in production environments is comparatively rare.
However, with the rise of digital business the number of organizations that provide additional capabilities at a higher cost within an application has steadily risen.
Rather than having to engage multiple vendors to provide that capability, Harness is now moving to invoke features management as code alongside all the other services built into its CI/CD platform, including AI models it is providing to accelerate the building and deployment of application at higher levels of scale.
Application development teams have, of course, been employing feature management since the 1970s to isolate the development of various components into a branch that can be worked on and tested without impacting the primary build the application is based on. Feature management, in effect, enables development teams to experiment with adding new capabilities in a way that doesn’t disrupt the application. When a project is completed, that branch is then usually merged into the primary build or can be deployed as a microservice that might be invoked via an application programming interface (API) by other modules with the application or another external application.
Feature Flags
At the core of that capability are what are known as feature flags, also known as feature toggles or feature switches, that make it possible to dynamically turn services on or off based on who is accessing them. Rather than being limited to being used in the development of applications, those feature flags are now being employed in production environments to enable the continuous delivery of multiple types of digital services to various classes of users of a particular application service.
Achieving that goal naturally requires DevOps teams to work closely with product managers to determine what capabilities will be made available to align with the business model adopted. The challenge, as always, is making sure those services provide a superior user experience without creating a set of dependencies that can’t easily be managed in isolation from the rest of the application.
vFunction, the AI-driven architectural observability platform today added real-time observability and GenAI assistant.
vFunction today added an ability to observe IT architectures in real-time to a platform that automatically converts monolithic application code into a set of microservices.
In addition, vFunction has added a generative artificial intelligence (AI) assistant that identifies issues and surfaces recommendations as changes are based to application environments. Based on the ChatGPT-4 service created by OpenAI, the vFunction Assistant is designed to make it simpler to remediate and refactor applications.
Leveraging OpenTelemetry
vFunction CEO Moti Rafalin said these capabilities all leverage OpenTelemetry, open-source agent software being advanced under the auspices of the Cloud Native Computing Foundation (CNCF) to collect telemetry data.
It’s not clear how many applications are instrumented using OpenTelemetry, however, a survey of 1,000 software engineering professionals conducted by vFunction suggests that as organizations look to streamline the management of application environments there is a clear need for increased visibility. A full 80% said these capabilities are either extremely or very valuable, with nearly four in 10 (39%) acknowledging they lack visibility into their IT architecture. A total of 40% said they advocate for a “shift left” approach to proactively address resiliency concerns earlier in the development lifecycle.
A similar percentage (41%) also plans to leverage generative AI to improve application performance and scalability, the survey finds.
The survey also finds more than half (51%) of respondents work for organizations that dedicate more than a quarter of their annual IT and engineering budgets to remediating applications, including refactoring and re-architecting efforts. More than three-quarters (77%) have implemented enterprise-wide initiatives to address technical debt.
Among those respondents, 57% work for organizations allocating more than a quarter of their IT budget to technical debt remediation involving monolithic applications, compared to 49% for microservices architectures. More than half 53% also noted their organization delayed major technology migrations or platform upgrades due to productivity concerns involving microservices. The survey also finds that 44% noted monolithic application complexity that results in a lack of clear domain boundaries and decreasing modularity is a major challenge.
Most application environments will be made up of monolithic and microservices-based applications for the foreseeable future. The challenge that organizations encounter is that as changes are made to application environments the level of drift away from the original architecture starts to increase.
Apply AI to Reduce Drift
At the same time, organizations accumulate technical debt as issues that should be remediated are left to be addressed at some future date. Unfortunately, many of those issues are never resolved, simply because new application development projects require DevOps teams to divert limited resources. However, as more applications are instrumented it becomes feasible to apply AI to reduce both drift and technical debt.
Each organization must determine to what degree they wish to modernize existing applications versus replacing or retiring them altogether. However, there are plenty of instances where legacy applications are still far too critical to be replaced. The challenge then becomes to what degree to update these applications, in a way that makes them more resilient and flexible as business requirements continue to evolve.
StarTree broadly enhances its real-time database platform by adding query support for metrics, logs and traces.
StarTree today made a case for becoming the foundation upon which observability is gained in IT environments by adding support for metrics, logs and traces to its real-time database platform delivered as a cloud service.
In addition, the company is also making generally available a StarTree ThirdEye service for detecting anomalies and identifying the root cause of application issues. It has also made a free serverless computing tier available on its cloud service.
Viable Alternative to the Legacy Database
Announced at the Real-Time Analytics Summit conference, StarTree Cloud is based on Apache Pinot, an open source columnar database optimized for real-time analytics, originally developed by LinkedIn. StarTree, via a private preview, is signaling that it views Apace Pinot as a viable alternative to the legacy database, upon which observability platforms DevOps teams use to troubleshoot IT environments.
StarTree, to further advance Apache Pinot, is also now adding a vector indexing capability to Apache Pinot along with support for a write application programming interface (API) to facilitate real-time synchronization with extract, transform and load (ELT) pipelines and additional integrations to Grafana and Tableau to improve visualizations.
Chinmay Soman, head of product for StarTree, said the issue with existing databases is they are not designed to scale to the level of performance that modern application environments will require.
Early Days for Achieving Observability
It’s still early days for achieving observability in application environments but many DevOps teams are now moving well past simply monitoring a set of pre-determined metrics. Observability platforms enable DevOps teams to launch queries that make it easier to troubleshoot applications. Not every DevOps team knows what queries to investigate. With the rise of artificial intelligence (AI), it’s becoming more feasible to rely on machine learning algorithms to analyze application environments using metrics, logs and traces stored in an observability platform.
Few DevOps teams will build an observability platform themselves but the capabilities of the underlying databases on they are based are critical, said Soman. DevOps teams should make certain the databases used in these platforms today will be able to scale to meet real-time processing requirements that will only steadily increase as more complex applications are deployed across distributed computing environments, he added.
In the meantime, StarTree is moving toward providing more observability capabilities via the managed database service it provides via the cloud.
It’s only a matter of time before DevOps teams need deeper levels of visibility to manage modern application environments in real time. As organizations become more dependent on software than ever, even the most minor glitch can now have a major financial impact. The challenge DevOps teams face, of course, is finding the funding needed to acquire these capabilities.
In the meantime, DevOps teams should at the very least start assessing which of the many monitoring tools being employed today might one day be rationalized by an observability platform, that in time might lower total costs by streamlining DevOps workflows that today are often more disjointed than anyone cares to admit.
Announced at a POST/CON 2024 conference, Postbot is included with version 11 of the Postman platform which also adds workspaces that make it simpler for software engineering teams to collaborate.
Postman CEO Abhinav Asthana said the overall goal is to improve the quality of the APIs being developed while at the same time reducing the level of friction that occurs as APIs are developed and updated.
Developers are already making extensive use of AI tools to write code, but Postman is using a combination of open-source large language models (LLMs) and LLMs from Open AI and Microsoft to automatically generate tests and documentation based on the data its platform collects. In addition to making it easier to create API tests using a conversational interface, the overall quality of those tests will improve using LLMs specifically trained using Postman data, noted Asthana.
Developers Need to Rely on AI Tools
That’s critical because the ratio between developers and the number of services that expose APIs is much too high, said Asthana. Developers will need to rely more on AI tools to create and manage APIs as the number of services they are expected to manage continues to steadily increase.
It’s not always clear who within organizations is managing APIs. Developers undoubtedly create APIs, but the management of APIs might be assigned to a DevOps team. As more organizations embrace platform engineering as a methodology for managing DevOps workflows at scale the more likely it becomes APIs will be managed centrally.
Ultimately, the rise of microservices-based applications has dramatically increased the number of APIs that need to be managed. APIs play a crucial role in making data available externally to other applications. However, the bulk of APIs being created today are for integrating all the microservices that make up a modern application environment. Those largely internal APIs are increasing at a pace that makes it challenging for individual developers to manage at scale. As a result, many organizations now have a significant API change management challenge, noted Asthana.
Each organization will need to determine to what degree the management of microservices is sustainable. Rather than getting caught up in philosophical debates, many organizations are deploying a mix of microservices alongside monolithic applications that also expose an API. In effect, monolithic applications are becoming large microservices. Regardless of approach, the number of APIs, including rogue and zombie APIs that organizations either didn’t realize were built or have simply been forgotten, is dizzying. In many ways, APIs in the absence of a management framework quickly become too much of a good thing.
There are multiple ways to manage APIs but the one thing that is critical to remember is the approach taken needs to integrate well with the tools used to create those APIs in the first place.
The emergence of low/no-code platforms is challenging traditional notions of coding expertise. Gone are the days when coding was an exclusive skill set reserved for a knowledgeable few. Low/no-code platforms have democratized software development. They empower individuals from non-IT or technical backgrounds to translate their business ideas into applications without the need to master complex programming languages.
These low/no-code platforms, with their user-friendly drag-and-drop interfaces and pre-built components, have simplified application development. Imagine building a smart home application with just a few clicks and drags — low/no-code platforms are making it possible.
Add generative artificial intelligence (Gen AI) into the mix, and the new development practices change the field remarkably.
The Impact of Gen AI on Low-Code Platforms
Expect low-code platforms to evolve further, becoming more intelligent, adaptable and seamlessly integrated into the software development lifecycle. There will be a shift towards platforms that use Gen AI to understand user intent, automate routine tasks and generate complex code snippets.
Gartner predicts that, by 2026, over 80% of enterprises will use GenAI APIs and models or deploy GenAI-enabled applications in production environments, up from less than 5% in early 2023.
Combining low-code and Gen AI will empower developers by automating routine tasks and allowing them to focus on high-level logic and creativity. Gen AI acts as a multiplier that can accelerate and optimize developmental operations.
McKinsey estimates that generative AI could add $2.6 trillion to $4.4 trillion annually to the global economy by 2040.
Another Tool: AI Copilots and Code Generators
Another promising application for AI large language models (LLMs) is AI-powered assistants. Enter copilots and code generators — AI assistants that take the hassle out of coding.
While low/no-code platforms have transformed application development, copilots and code generators add an extra layer of efficiency.
AI copilots use a chat interface to simplify coding through a chat interface, where developers articulate their code requirements in natural language prompts, and the copilot generates the code accordingly. These assistants provide real-time assistance, offering contextual suggestions and code snippets, and they can anticipate potential bugs. Moreover, they can explain code blocks, generate unit tests and propose bug fixes, which enhances the coding process.
AI copilots are like trusted assistants, ready to aid you with specific coding tasks or generate complex code – and they do it swiftly.
The Big Question
As a result of low/no-code platforms gaining traction, the skill sets required in the tech industry will transform. Gone are the days when proficiency in multiple programming languages was a must-have.
With low/no-code tools taking center stage, does this mean coding is becoming obsolete?
A big no.
While basic programming knowledge remains valuable, it’s no longer the sole requirement for building innovative applications. With low/no-code platforms democratizing application development, individuals from diverse backgrounds can leverage technology to create impactful solutions.
Do GenAI, copilots, and code generators render low/no-code platforms obsolete?
Again, no.
While GenAI, copilots, and code generators excel at certain specific tasks, low/no-code platforms offer a holistic solution, combining visual interface design, database integration and more. They’re the Swiss army knives of development, catering to a wide range of needs beyond just code generation.
Gen AI, copilots and code generators are not here to replace low/no-code platforms or traditional coding practices. Instead, they are positioned to complement and enhance them. The fusion of these technologies holds great promise in simplifying software development, boosting efficiency, and unlocking new avenues for innovation.
The Final Word
With better code suggestions and automated testing capabilities, low/no-code platforms deliver high-quality solutions that meet the demands of today’s tech-savvy users.
With the thrilling combination of low/no-code platforms and GenAI, the future of software development is promising. While these advancements may disrupt traditional coding, they also present opportunities for innovation and creativity.
It’s time for us to embrace this change and build a brighter tomorrow, one line of code — or lack thereof — at a time.
Achieving full observability across enterprise applications is challenging. While third-party services like New Relic or Datadog make end-to-end observability easier, as your application grows in complexity, so does the telemetry data, leading to increased costs. Migrating to an open source stack gives you control over telemetry data and reduces observability costs, despite challenges with existing service provider commitments.
Migrating to open source solutions is straightforward with simple architecture and tech stacks. However, transitioning the entire observability stack to open source demands meticulous planning, tools/frameworks assessments, testing, risk analysis, and cross-team communication, especially given the number of microservices, heavy cloud reliance, and diverse languages and frameworks to take into account.
A cloud observability platform usually absorbs 20%-30% of overall infrastructure spending, but in some cases, it can reach 50%-60%. If your observability expenses exceed 50%, then transitioning to open source solutions would be a sensible option to mitigate tech infrastructure costs.
For the past two years, I have been working on migrating the observability platform from a proprietary solution provider to an open source stack. Let’s dive into the key steps necessary for such a migration.
Finalize the key telemetry data and systems
Managed observability platforms offer comprehensive insights into system health, necessitating a significant volume of high-quality telemetry data. By default, the agents tasked with capturing and transmitting telemetry data are configured to gather as much information as possible, facilitating the creation of extensive dashboards and reports. However, this extensive data collection contributes to the overall cost.
Select which systems you actually need to monitor. A typical enterprise application includes databases, caching databases, container orchestrators, and numerous cloud services. Engineers often monitor more services than necessary, which leads to unnecessary complexities and costs. It’s often sufficient to streamline the monitoring scope to essential systems.
In my experience, the quantity of telemetry data required is significantly lower than what third-party services typically retrieve. You can easily divide those data collections into two buckets: must-have and nice-to-have. The “must-have” data aligns with the well-established four golden signals:
Latency: the amount of time it takes to serve a request
Traffic: the volume of requests that a system is currently handling
The error rate: the number of requests that fail or return unexpected responses
Resource saturation: the percentage of available resources being consumed
These fundamental signals serve as the foundation for key dashboards and alerts. Observability platforms built on these signals cover 80% of typical use cases.
Understanding each signal’s metadata requirements is crucial. The extent of metadata captured directly impacts the observability platform’s complexity and cost. For example, when monitoring service-to-service latency, consider the necessity for service IP addresses, instance IDs, or header information.
Select the relevant stack
Managed observability platforms such as New Relic and Datadog provide comprehensive monitoring for various enterprise components. However, when you migrate to an open source stack, you must assess and integrate the tool stacks to meet diverse monitoring requirements.
One important aspect to consider is scaling: how to manage the huge amount of data generated every minute across all those systems. Focus on two fronts: selecting the stack for storing and processing telemetry data (logs, metrics and traces) and devising methods for capturing and pushing telemetry data from diverse systems.
Telemetry signal stack selections
Stack for logs
You require a stack that can efficiently and economically process and store the substantial volume of logs generated by the system. In the past, I used the ELK stack for storing and searching logs, but it is a generic solution and not tailored specifically for logs.
I recommend Grafana’s Loki for its effective management of large log volumes and LogQL, a language akin to PromQL. If you are familiar with PromQL, navigating through extensive log data with LogQL is easy and intuitive.
Stack for Metrics
Prometheus is a popular database for storing time-series metrics data. However, it has scaling-related limitations; it cannot scale horizontally. Alternatives like Thanos, Grafana’s Mimir, and VictoriaMetrics offer better out-of-the-box solutions for horizontal scaling.
I conducted some research and ultimately chose Grafana’s Mimir to store metrics in a similar migration project. This decision was based on Mimir’s remote storage capability, as well as its scalable and highly available architecture.
Stack of Tracing
Distributed tracing is a must in a microservices architecture, for easily identifying bottlenecks in latency and for resolving performance-related bugs. Grafana’s Tempo could be a good option for several reasons:
It can seamlessly integrate as a backend for Grafana’s dashboard. Consolidating dashboards in Grafana helps avoid navigating between multiple applications. Switching platforms just for tracing while using Grafana for other aspects of observability would not be practical.
Tempo provides a similar experience to TraceQL when interacting with the database, streamlining the learning process and significantly reducing the learning curve.
Tempo is compatible with popular open source tracing protocols such as Zipkin and Jaeger. If any team already uses these protocols, transitioning to Grafana’s Tempo would be smoother.
Capturing and pushing telemetry data
You have to install an agent on the system to capture and push the telemetry data. Those steps vary based on the system being monitored. Typical enterprise application setups involve monitoring cloud integrations, process monitoring, infrastructure hosts, Kubernetes clusters and application monitoring. Among the possibilities:
Cloud integrations: This includes monitoring for cloud services such as SQS, SNS, EMR and EC2.
Process monitoring: This involves monitoring processes running on bare-metal machines. With the advent of dockerization and Kubernetes, there is now a standardized way of starting the application. In the past, there was no fixed mechanism. For example, to run a Java application, you could use a Java application, the ‘java -jar’ command, Tomcat, or OS systemctl. In the case of node, it could be npm or PM2. Each team or service may have had its own way of starting the process.
Infrastructure host: This entails monitoring the machine itself for metrics such as CPU usage, memory, disk I/O and Network IO or checking if the machine is offline.
Kubernetes monitoring: You regularly need to monitor a Kubernetes cluster, such as instances where Kubernetes is unable to schedule a pod due to insufficient resources.
Application monitoring: This monitoring focuses on overseeing the services the team created. Each service may differ in its development approach and choice of tech stack; but from an observability perspective, they usually are treated the same.
Browser and mobile monitoring: These metrics ensure optimal performance and user experience across different platforms. Browser monitoring includes tracking page load times, rendering performance, JavaScript errors and resource usage. For mobile, it includes monitoring app crashes, latency, battery usage, network requests and device-specific metrics.
Before the advent of OpenTelemetry, there was no standardized method for monitoring applications. OpenTelemetry emerged from the merger of two earlier projects, OpenTracing and OpenCensus. It stands as a vendor- and tool-agnostic framework to instrument applications or systems irrespective of language, infrastructure or runtime environment. OpenTelemetry represents a significant community effort, and its popularity and stability are steadily growing.
Validate observability stack on application architecture
Applications vary in architecture and development stages. To address this diversity, you need multiple proofs-of-concept (POCs) across different systems, such as infrastructure, cloud service and backend and frontend applications.
Migrating an observability platform from one tech stack to another is simple when the application architecture is a monolith. However, when the underlying architecture is microservices, it becomes difficult.
While the microservices architecture offers flexibility in choosing different tech stacks for building services, devising a standard solution for capturing telemetry data becomes challenging due to the diverse range of tech stacks involved.
Conducting POCs for various tech combinations is necessary to build a self-service tool that individual teams can follow to migrate their services easily. For instance, if your service uses Java 11, Spring Boot 3.x.x and PostgreSQL, these POCs can provide standardized steps to enable application monitoring.
Migrate core components
Setting up the observability platform involves configuring the stack for metrics, logs and traces, as well as installing the necessary agents. One advantage of migrating the observability platform is that there’s no need to transfer old telemetry data. During the testing phase, you will have an overlapping period to accumulate enough useful data to create specific dashboards and alerts.
Migrating alerts and dashboards to the new system is essential, even when data migration is not necessary. This is because changes in the underlying techniques for capturing metrics may modify the names of the metrics and subsequently change all query expressions used in alerts and dashboards. Some metrics may not be available in certain scenarios, especially derived metrics that can be recreated from underlying metrics.
While migrating query expressions manually is an option, a manual process is error-prone and time-consuming for a large number of alerts. I had experienced a similar situation with alert numbers ranging from 100 to 400.
To streamline the migration, we developed a Node.js script that programmatically converted New Relic’s query expressions to Prometheus expressions. The script performed the following high-level steps:
Connected to the New Relic API server and fetched all the alerts configured for cloud integration
Converted New Relic alert expressions into PromQL alert expressions
Wrote all the Prometheus alert expressions into the YAML file
Implementing this technique allowed us to successfully migrate all alerts and dashboards in four or five days, a process that would normally demand a month’s manual labor. Additionally, scripting significantly minimized the risk of human error.
Test the migration
Verify each alert and dashboard. Run both old and new platforms during testing. Parallel operation permits thorough testing of the metrics.
However, running both systems concurrently may lead to a performance hit due to services sending telemetry data to two locations. Additionally, there may be a slight increase in costs until the complete transition.
Running both systems in parallel provides an ideal setup to ensure the migrated dashboards and alerts are functioning correctly.
Testing alerts that trigger during this process is straightforward, but evaluating those that have not yet triggered poses a challenge. In one project, I reduced the threshold value of each alert and tested its functionalities. Approximately 90% of the alerts generated through custom scripts worked seamlessly, with only 10% requiring some manual tweaking.
Migrate other related components
Other systems depend on the data generated by the observability platform. For example:
Notification systems like email, Slack channel, etc., that alert in case of an incident
Incident management tools like PagerDuty provide a streamlined way to handle incidents.
These systems rely on the alert’s payload to work properly. Their assets should be updated, as a change in the observability platform leads to a change in the alert payload. Update the templates of notification systems to ensure they seamlessly integrate with the new alert payload.
Similarly, incident management tools like PagerDuty require modifications to routing rules, escalation policies, and scheduling. Fortunately, readymade open-source migration tools are available to facilitate easy migration from PagerDuty to Grafana’s OnCall. However, in other solutions, you may not have access to out-of-the-box migration tools. In such cases, manual migration or script writing may be required.
Timeframe for the Migration
Planning a migration involves finalizing key telemetry data and systems, selecting relevant stacks. and validating the observability stack on the application architecture. It typically takes around three weeks.
The project duration depends on factors such as the number of teams involved and the quantity of active services. For instance, a migration to open source with 100 microservices and involvement from 10 different teams may take four months to complete.
Migration Complete!
Transitioning your observability platform to an open-source stack offers a promising avenue for cost reduction and increased control over telemetry data. However, accomplishing this migration demands meticulous planning and execution, encompassing essential steps like feature prioritization, stack selection, POCs, core component migration, testing, and the migration of related systems. Despite the challenges posed by diverse architectures and technologies, a systematic approach, comprehensive assessment, and collaboration among teams can facilitate a smooth migration process and guarantee its success.
Julianna Cammarano, vice president of product for applications at Rocket Software, said the update makes it possible for DevOps teams to apply policies that prevent developers from being able to view sensitive data, such as credit card or social security numbers that might result in a compliance violation.
The masking capability makes it possible for DevOps teams to continue to provide access to data that is needed to test applications without risking potential fines that might be levied if they fail an audit, she added. In the absence of that capability, organizations that must comply with any number of regulations are not able to give developers access to data sets running in production environments to test their applications, said Cammarano.
As is the case with most legacy IBM platforms, adoption of best DevOps practices is uneven. Many organizations are now using a mix of waterfall and DevOps methodologies to build and deploy applications. In some cases, an application that is rarely updated lends itself more to a waterfall methodology than a DevOps workflow that is designed to enable organizations to frequently update software.
The challenge that organizations with iSeries applications have is attracting developers and software engineers to work on a legacy platform, noted Cammarano. The only way to entice them to work on a legacy platform is to make sure they have access to modern DevOps tools that can be easily accessed via a centralized portal, she added.
It’s not clear how many new applications are being built for the iSeries platform but there are still a substantial number of legacy applications running in production environments that organizations may opt to modernize rather than replace, especially if the cost of replacing those applications is prohibitive, said Cammarano.
IBM, for its part, appears committed to providing support for the iSeries platform till 2032, and beyond, now that IBM is using the same server architecture developed for its P-Series Unix platforms to build next generation iSeries servers. That approach ensures iSeries application performance will continue to improve as IBM makes additional investments in the RISC processors used now in both platforms.
There are, of course, few organizations that are not also building and deploying applications on other platforms. Those that do have iSeries platforms are typically looking to adopt a consistent set of DevOps workflows that can be applied to multiple platforms that continue to run in on-premises IT environments, mainly because migrating iSeries applications to another platform requires a significant amount of time and effort.
Ultimately, each organization needs to determine what level of DevOps adoption best fits their needs, depending on how dependent they might be on a specific platform. The one thing that is certain is many organizations, as more application development and deployment processes become automated, will discover DevOps is a journey that never ends.
At a GrafanaCon event today, Grafana Labs unveiled an update to its core Grafana visualization platform. The company is offering an Explore Metric module which it says can identify an issue’s root cause — and to do so without anyone having to launch a query against the open source Prometheus platform that is at the core of the Grafana Cloud observability platform.
That’s not the only enhancement. Version 11 of Grafana also makes it easier to edit and share dashboards, in addition to streamlining the number of alerts that a single incident might generate. In addition, Grafana Labs is now making available a curated edition of the OpenTelemetry software dubbed Grafana Alloy that it will extend and support alongside the rest of its open source software portfolio.
Finally, the company is updating Loki log management to make it simpler to streamline the storage of log data.
Grafana Labs CTO Thomas Wilkie said that collectively these updates will make it easier for a wider range of IT professionals, including application developers, to invoke the capabilities of a Grafana Cloud platform based on open source software.
As IT environments become more complex, the need for observability platforms becomes more pronounced. IT teams typically have access to a raft of monitoring tools for tracking pre-defined metrics but their ability to launch queries to surface the root cause of an issue has been limited. As observability platforms continue to evolve, the line between observability and monitoring continues to blur, said Wilkie.
Less clear is to what degree observability platforms might enable IT teams to rationalize existing monitoring tools. In the meantime, the pace at which IT teams are embracing observability varies widely from one organization to another. IT teams need enough expertise to launch relevant queries to get the full return on their investment in an observability platform. In the absence of that expertise, acquiring an observability platform becomes less compelling, in the short term.
In theory, there soon should come a day when machine learning algorithms and other forms of artificial intelligence (AI) make it easier for organizations to identify issues without having to craft queries. But it’s not likely AI will eliminate the need for site reliability engineers (SREs), said Wilkie. Instead, AI will make it simpler to onboard new members to a DevOps team while deploying more applications than ever, he added.
It’s still early days so far as adoption of AI and observability is concerned but there will be an inevitable convergence. The overall goal should be to make it easier to troubleshoot highly distributed application environments that today span cloud, on-premises and edge computing applications. That’s critical because as application environments become more distributed the overall size of most of the IT teams tasked with managing them is not increasing. As such, that pace at which DevOps workflows can be automated will prove crucial.
In the meantime, IT teams should expect it will become more difficult to triage complex IT environments before it hopefully becomes simpler thanks to the rise of observability, automation and AI.
Most IT teams regularly employ a range of tools to monitor application performance. But in an era where applications are running everywhere from the cloud to the network edge, the visibility being provided is limited. The metrics and analysis being surfaced are roughly akin to only being able to see a few yards ahead when driving a vehicle on a rainy night. Managing modern applications instead requires platforms that provide the equivalent of radar and sonar capabilities, so that IT teams have better visibility into application environments that today remain far too shrouded to effectively manage.
Achieving that goal requires a deeper insight into how various Internet services impact application performance. Just about every application depends on the quality of one or more of those services. The more application services that depend on the Internet, the more probable it becomes that someone will encounter issues intermittently impacting performance in ways that are difficult to understand, much less predict.
Unfortunately, most IT teams today have limited ability to discern how the performance of Internet services is impacting their applications. There are, of course, Internet performance monitoring (IPM) tools capable of surfacing network performance metrics. The challenge and opportunity now is to surface those metrics in context with all the other telemetry data that DevOps teams collect from the various application performance management (APM) and observability platforms they rely on to monitor and troubleshoot application environments.
Why Applications Are Becoming More Distributed
First, with the rise of cloud computing followed by the proliferation of software-as-a-service (SaaS) applications and now the rise of edge computing, applications are becoming more distributed with each passing day. The challenge is that each application is invoked across a network that adds latency, and existing APM and observability platforms cannot effectively analyze it.
The pace at which applications are becoming more distributed is only going to increase. More data is being processed and analyzed at the network edge as organizations deploy near real-time applications. Instead of transferring raw data to the cloud, these applications process and analyze data at the point where it is created and consumed. It’s only then that an application either streams or passes the aggregated results to a cloud service or to an on-premises IT environment for further analysis.
Most DevOps teams, however, lack visibility into how Internet services impact more highly distributed applications, beyond the cloud service or on-premises IT environment where they historically deployed their applications.
Internet Blind Spots
Broadly, there are three major classes of blind spots that impact distributed application performance. The first and arguably most opaque are the services provided by third-party vendors. Ranging from content delivery networks (CDNs) to software-as-a-service (SaaS) application, each of these services is controlled by an external service provider that typically doesn’t allows a DevOps team to collect telemetry data by deploying agent software in their IT environments. At best, they may expose an application programming interface (API) to enable an agentless approach to collecting data, but that method doesn’t typically provide the level of control required to optimize application performance.
The second major blind spot is at the device level. Configuration errors in browsers can have a significant impact on loading times that adversely impacts application performance, for example.
Finally, various platforms and protocols ranging Domain Name Servers (DNS) to publish-and-subscribe services can impact applications in way that an APM or observability platform isn’t going to detect.
The only way to provide the level of visibility required is to leverage an IPM platform that collect telemetry data from a global network, made up of thousands of points-of-presence (PoPs) to determine precisely what level of Internet service is delivered to any given end point.
Context and Control
Integrating the telemetry data collected via an IPM provided DevOps teams with a context to better troubleshoot distributed applications. Instead of wondering if a certain piece of code is to blame, a DevOps team can immediately determine whether Internet traffic moving through a specific location is either creating additional latency or blocking access altogether.
Armed with those insights, the team can contact then Internet services provider to address the issue or, if necessary, reroute via another service to ensure application availability.
Regardless of approach, the days when DevOps teams found themselves subject to the vagaries of network services, over which they had little to no influence, are coming to an end.
DevOps Meets Netops
The integration of IPM and APM or observability platforms creates a foundation upon which best practices for managing network operations (NetOps) and DevOps can finally be melded. Instead of having to convene “war rooms” to uncover the root cause of an issue, both teams will instantly be able to visualize to what degree networking services might be adversely impacting networking services. All that time spent guessing what the cause of a problem might be can now be allocated to remediating the issue.
In an era where applications have never been more dependent on networking services, the need to converge the management of NetOps and DevOps workflows has never been more critical.
Conclusion
Most DevOps teams know all too well there are often major discrepancies between what DevOps teams relying on APM and observability platforms believe and what is actually occurring in a distributed computing environment. The challenge is that they typically lacked the visibility required to determine what best course of action to address the issue. As IPMs become more integrated with APM and observability platforms, network blind spots that previously made discovering the root cause of an issue will soon be all but eliminated.
Of course, each DevOps team needs to determine to what degree gaining those insights is a priority. However, given the number of issues DevOps teams are trying to resolve that can be traced back to an Internet service, it’s clear the overall level of stress and toil DevOps teams regularly encounter could be greatly reduced by simply providing more networking visibility into those services.
