devopsy.kinsta.cloud

Category: Blogs

Does Platform Engineering Make DevOps & DevSecOps Irrelevant?
For those of us old enough to remember, the DevOps movement came onto the scene somewhere around 2007/2008. Its proponents railed against the traditional software model at the time, which called for the developers who wrote the code to be organizationally and functionally separate from those who deployed and supported the code. The reasoning behind the movement is that these siloed teams, concerned only with their own objectives, resulted in botched releases, significant delays, poor communication and, ultimately, unhappy customers. It took a few years to gain traction, but when Gartner got on board with the concept in 2011, adoption soared, and Agile development practices like automated build and test and continuous integration and delivery became the new norm.

Security and Compliance Join the Mix

Agile, DevOps and its promise of continuous integration and continuous delivery … well, delivered. Development and operations teams were finally on the same page, each able to see a current phase’s end goal, providing the flexibility to make changes and improvements along the way, speeding the delivery of quality software. Unfortunately, in these early years of DevOps success, security and compliance generally remained siloed. That silo resulted in security being an afterthought—stuck near the end of the application development life cycle. Consequently, security took the blame for release-delaying bottlenecks.

The year 2014 was notable for an exponential increase in security breaches compared to the year before. According to an article in Security Week, 2014 marked the year that, for the first time, one billion records were compromised in more than 1,500 notable data breaches, an increase of 80% from 2013. Home Depot, JP Morgan Chase and eBay were among the many exposed to attackers. Given these and other high-profile breaches, it was time to re-evaluate development processes and give security and compliance a seat at the table.

With research by Microsoft and others showing that 80% of security breaches were related to configuration errors, the concept of DevSecOps, putting security front and center in the Agile development process, ensures that assets are configured correctly in the first place, providing continuous compliance by running constant scans to identify configuration drift.

This approach can be extended beyond the data center to any cloud to any edge, where even applications can be managed and checked against compliance and security standards. You can even extend a single DevSecOps compliance framework to cloud native assets including Kubernetes and public cloud services.

Today, it is safe to say that the term “DevOps” is likely losing its relevance, but “DevSecOps” is alive and well, as long as the organization embraces the cultural, people and process changes necessary for success. Tools and technology are essential to DevSecOps, as is adopting a culture that security is everyone’s responsibility. That culture requires instigating a security-first mindset among teams as well as implementing essential automated security testing tools.

What About Platform Engineering?

Platform engineering has recently captured interest and is now generating considerable buzz due to its focus on building and operating self-service internal developer platforms (IDPs) for software delivery and life cycle management. The platform is supported by layered services or tools, created and maintained by a dedicated product team, designed to support the needs of software developers by essentially stitching together components to create a frictionless developer experience.

As Gartner says, “Platform engineering is an emerging technology approach that can accelerate the delivery of applications and the pace at which they produce business value.” In fact, Gartner expects “[t]hat by 2026, 80% of software engineering organizations will establish platform teams as internal providers of reusable services, components and tools for application delivery. Platform engineering will ultimately solve the central problem of cooperation between software developers and operators.”

In that context, platform engineering certainly builds upon the Agile concepts of DevOps and DevSecOps. Looking deeper at DevOps/DevSecOps and platform engineering shows that they both share many characteristics, while they can also be combined in a way that provides greater benefits to organizations that think about them in tandem. It is easy to see the integral link between platform engineering and DevSecOps and the broader infrastructure management landscape. For example,
- Developer Experience and Productivity – DevSecOps extends experience and productivity support beyond the developer to IT Ops, security and compliance.
- Automation – Automating infrastructure configuration as well as compliance tasks using a policy-as-code approach is the lynchpin of DevSecOps. Platform engineering provides the opportunity to extend automation to the other parts of the development lifecycle.
- Self-service – This is another common concept where platform engineering can build off of the automation and services-oriented approach that DevSecOps uses to not only enable developer self-service for code/test/deploy, but also to enable developers to be more proactive and effective in the security/compliance realm.
- Accelerate Value – Accelerating value is the focus of any tech enabler, from platforms to tools and processes.
From an Agile development approach supported by DevOps/DevSecOps and extending now to platform engineering, the end game is all about speeding delivery of high-impact applications with continuous updates that are highly resilient, reliable and secure. Given the synergies and the pressure on organizations to rapidly deliver applications while combatting cyberthreats, any platform engineering effort should ensure that DevOps/DevSecOps plays a central role.

While the term DevOps and, to some extent, DevSecOps, sound a bit dated and irrelevant and could maybe use a new moniker, the evolving role they are certain to play as platform engineering gains a foothold are not to be underestimated.
August 31, 2023
Grafana Labs Delivers Open Source Code Profiling Tool

Grafana Labs this week released version 1.0 of an open source tool that collects data about an application’s code to aid debugging, troubleshooting and optimization efforts. The tool makes it simpler to identify the specific lines of code driving a process or task within an application.

Ryan Perry, engineering director for Pyroscope at Grafana Labs, said Grafana Pyroscope 1.0 combines a tool acquired last year from a company he led with Grafana Phlare, a code profiling tool that Grafana Labs previously developed. The combination of the two tools provides a more precise code-tracking capability with an underlying database that is designed to scale horizontally, he added.

For example, Grafana Pyroscope 1.0, which supports extended Berkeley Packet Filtering (eBPF) running on Linux systems, can now ingest, store and query workloads from any project of any size. Grafana Pyroscope 1.0 has already been used to profile more than 50,000 application instances at a rate of more than 6,000 unique profiles per second.

The code profiling tool was added to Grafana Cloud Profiles, the hosted continuous profiling tool that Grafana makes available as a service, earlier this month.

While the primary use case for Grafana Pyroscope is to troubleshoot applications, organizations are also using it to identify and then remove dead code within an application as part of a larger application modernization initiative, noted Perry. As updates to applications are made over several years, organizations frequently find that subsets of the application code are no longer needed, said Perry.

The goal going forward is to add support for features such as flame graphs in GitHub pull requests, canary deployments, profile-guided optimization (PGO), integrating profiling with load testing tools such as Grafana k6 and integration with visual editors such as VS Code.

In addition, Grafana Labs is committed to working with the OpenTelemetry project led by the Cloud Native Computing Foundation (CNCF) to ensure Grafana Pyroscope remains aligned with the OpenTelemtry effort to develop an OTel-native profiling tool. Grafana Pyroscope 1.0 will provide the ability to ingest data from that tool and contribute client-side instrumentation capabilities the company has developed.

Grafana Labs is also working toward improving client-side software development kits (SDKs) to add more real-time agent management features, such as sampling controls, ad hoc profiling capabilities and alternative instrumentation options.

Finally, Grafana Labs plans to improve eBPF integration for languages such as Python, Java and .NET.

DevOps teams naturally spend a lot of time trying to optimize application performance and troubleshooting issues that arise long after an application is initially deployed. Code profiling tools are emerging as a tool that makes it much simpler to aggregate metrics, logs and traces and surface actionable insights that make debugging and troubleshooting applications much less tedious. The challenge is finding the time to identify and proactively address potential issues before they become another incident to manage.

August 31, 2023
Product Mindset Key to Platform Engineering Success

Platform engineering has taken the world by storm, and for good reason. Good platforms reduce TicketOps, standardize config setups, and cut lead time and time to market. Despite the discipline’s new popularity, there remain some questions about what sets platform engineering apart from DevOps. The answer to those questions lies in the platform as a product approach–the heart of every successful platform engineering effort.

What is Platform Engineering?

According to Luca Galante, platform engineering is “the discipline of designing and building toolchains and workflows that enable self-service capabilities for software engineering organizations in the cloud-native era.” Platform engineers build an internal developer platform, which “consists of many different tech and tools, glued together in a way that lowers the cognitive load on developers without abstracting away context and underlying technologies.”

What is the Platform as a Product Approach?

A product approach requires user research, a defined mission, leveraging lighthouse teams, getting regular feedback, iterating and marketing your platform for internal buy-in. Together, these product management best practices will ensure your platform enables true developer self-service and finds the right level of abstraction.

“Build it, and they will come” is not a viable strategy for ensuring developers will adopt your platform. Similarly, forcing developers to use a platform is likely to fail. Mandated usage creates resentment, reduces trust and results in shadow ops. To avoid your platform’s untimely demise, you have to treat your platform like a product and sell it to your customers: Your developers.

This article provides a high-level overview of what a product mindset looks like in practice and why each component contributes to the success of your platform.

Do Your Research

Unless you gain a comprehensive understanding of developers’ pain points and what they’re already doing to mitigate those issues, you won’t build a platform people actually want to use. Hashicorp’s Michael Galloway shared this example of a user survey he conducted with his team while at Doma. The idea is to ask questions about the full software delivery process, from the planning stages of the interviewee’s work to debugging and testing. This way, the platform team can prioritize creating features to solve developers’ most frustrating problems.

Oftentimes, platform teams will find that teams have conflicting wants and perspectives on what the platform should be. A platform product manager will help translate disparate responses into an actionable and successful plan. Successful platform teams also research open source and commercial tools and integrate pre-built solutions into their platform where beneficial. They study blueprints, like the internal developer platform reference architectures developed by McKinsey, as a starting point.

Define Your Mission

Your mission statement gives your platform team a clear identity and priorities. It’s also a starting point for understanding and communicating your impact on the business to relevant stakeholders. Good mission statements are emotional and inspiring, simple but meaningful, match the longevity of the organization and are informed by user research.

Leveraging Lighthouse Teams

When starting their platform engineering journey, many organizations don’t know what to build first. The confusion arises from having two important yet conflicting priorities. On the one hand, your platform team doesn’t want to bite off more than it can chew by building too many features at once. On the other hand, platforms that are thin enough to cover the entire enterprise often fail to provide enough value to individual teams.

The way to strike the right balance is by leveraging lighthouse teams. A lighthouse team is the first group of people you’ll choose to build your organization’s platform for, so spend a lot of time with them to get it right. Cultivate ambassadors and evangelists and give them time to get the rest of the team (and the organization!) on board with your platform. Once you’ve built a platform that provides real value to your lighthouse team, repeat this process with more teams across the organization.

The lighthouse team approach is valuable because it helps platform teams avoid boiling the ocean while also ensuring your platform provides real value every step of the way.

Stay in the Loop

Initial user research is not enough to sustain a successful platform engineering effort. Platform teams should solicit regular feedback from developers and use it to iterate the platform.

Market Internally to Get Buy-In

Conversations about platform engineering often focus on developers, but developers are not your platform’s only stakeholder group. Your platform team will also need buy-in from managers, SysAdmins, DevOps engineers and the C-suite.

In his PlatformCon 2023 talk “How to communicate the business value of platform engineering,” Gartner’s Manjunath Bhat shared Gartner’s Enterprise Value Equation as a foundation for platform teams to understand and communicate the value of their Internal Developer Platform to different stakeholder groups.

The process includes assessing stakeholder priorities and concerns, identifying and defining value enablers, building a value map to map value enables to stakeholder impact, supporting the value story through outcome metrics, and communicating the “why” and realized value to the organization.

Sounds Great. What’s the Catch?

Applying a platform as a product approach can be difficult in practice. For example, many organizations struggle to afford the internal developer platform the same considerations as they would an external product. When working with internal developers, it’s easy for platform teams to make assumptions about what those users know, want and need. However, platform engineering requires avoiding these assumptions and always seeking user perspectives.

Another common pitfall organizations encounter is mandating platform adoption. Making the platform mandatory closes off the critical feedback loops platform teams need to gauge the success of their platform and make continuous improvements.

Conclusion

The platform as a product approach is what sets platform engineering apart from DevOps. It’s deceptively simple but important if you want to build an internal developer platform people actually want to use.

August 31, 2023
Google Adds App Modernization Tools to Generative AI Platform

Google this week added automated refactoring capabilities to the generative artificial intelligence (AI) tools it makes available to write code on Google Cloud.

In addition, Duet AI for Google Cloud is now integrated with Google’s Apigee API Management and Application Integration services. This makes it possible to design, create and publish application programming interfaces (APIs) using simple natural language prompts.

These additions to the preview of Duet AI for Google Cloud were announced at the Google Cloud Next 2023 conference and are scheduled to be generally available later this year. The new capabilities promise to reduce the time and effort required to modernize applications.

For example, DevOps teams can now convert code written in C++ into Go and migrate that application to Cloud SQL, the managed relational database service that Google provides. Via a natural language interface, these tasks are launched via a prompt from directly within a development environment.

Priyanka Vergadia, a developer advocate for Google, told attendees these capabilities will reduce the need to rely on consultants to migrate applications and take up much less time. In addition, the overall quality of the code should improve using large language models (LLMs) that have been trained using code written by Google developers and best practices defined by Google software engineers.

Organizations will save time and effort because the need for developers to understand how existing application code was constructed is eliminated, she noted.

Google has also started to work with select enterprise IT teams to allow Duet AI to incorporate knowledge from their libraries and codebases to generate more context-aware code suggestions.

In addition to writing code, Google is making a case for also using Duet AI in Google Cloud to monitor performance and troubleshoot IT issues by identifying correlations across application environments. For example, natural language prompts can be translated into PromQL queries to analyze time-series metrics. Duet AI can also provide intuitive explanations of complex log entries in Logs Explorer for easier root-cause analysis and provide suggestions for how to fix issues that surfaced in an error reporting tool.

Duet AI is also incorporated into the Google BigQuery service to help developers write SQL and Python code to access and analyze data. It can generate full functions and code blocks, auto-suggest code completions and explain your code and SQL queries. Via a single SQL statement, a query can connect tables with the foundational AI models Google makes available via its Vertex AI service.

At the same time, IT teams can optimize prompts using BigQuery Studio tools, perform text analysis or generate new attributes to enrich a BigQuery data model. Duet AI also enables IT teams to generate vector embeddings in BigQuery to create semantic searches and recommendation queries.

It’s not clear whether generative AI capabilities will automate entire DevOps and DataOps workflows just yet, but many of the tasks that once required automation scripts will soon be automated by AI models embedded within DevOps platforms. The challenge and the opportunity is to determine how to make those capabilities available to drive DevOps workflows at unprecedented scale.

August 30, 2023
CDF Report: Steady Growth in Number of Jenkins Jobs Run

The Jenkins project that operates under the auspices of The Continuous Delivery Foundation (CDF) reported this week that over a two-year period, usage of pipelines grew 79%, with total workloads defined in the continuous integration/continuous delivery (CI/CD) platform growing 45% over the same period.

Based on analysis conducted from June 2021 to June 2023, the report found the number of jobs defined increased from 27,105,176 jobs to 48,625,398 jobs per month. The number of all jobs defined on Jenkins increased from 50,785,205 to 73,746,418 jobs per month (45%).

Sacha Labourey, a member of the governing board for the CDF and chief strategy officer at CloudBees, a provider of commercially supported instances of the open source Jenkins CI/CD platform, said it’s clear that more applications are being constructed by more organizations than ever using DevOps workflows.

At present, the CDF reported there are now more than 600 active contributors to the Jenkins project. That level of support has made it possible for Jenkins to become a standard CI/CD platform. DevOps teams know there is a substantial open source community they can count on for support, said Labourey.

The challenge now is how to centrally observe DevOps workflows to accelerate application development while at the same time giving developers more freedom to innovate. Platform engineering principles represent an opportunity to achieve that goal, as long as developers still have an opportunity to innovate and collaboratively address bottlenecks in DevOps workflows as they occur.

The platform engineering team should be observing processes and proactively making recommendations and define a set of guardrails that ensure DevSecOps best practices are followed, he added.

Otherwise, application developers are likely to view platform engineering as another effort by centralized IT teams to take power away from the developer, noted Labourey.

Striking that balance is especially critical as the number of cloud-native applications being built and deployed alongside classic monolithic applications has significantly increased, he added. DevOps teams need to be able to centralize observability of what are two distinct styles of application development that tend to occur at a significantly different pace from one another, Labourey said.

Hopefully, advances in artificial intelligence (AI) will make it easier for DevOps teams to strike the right balance between management and innovation. In the meantime, however, as the pace of application development continues to accelerate, each DevOps team will need to determine how best to evolve their existing workflows. That’s especially critical at a time when economic headwinds are requiring organizations to find ways to increase existing developer productivity rather than hire or contract additional developers.

In the meantime, it’s apparent that DevOps, as a methodology for building and deploying applications faster, is here to stay. The next major challenge is finding a way to achieve that goal at levels of scale that require automation everywhere from the network edge to the cloud.

August 30, 2023
Gremlin Adds Detected Risk Tool to Chaos Engineering Service

Gremlin today added a risk detection capability to its chaos engineering service that automatically identifies issues that could cause outages along with recommendations to resolve them.

Enabled by agent software deployed in an IT environment, the Detected Risk tool makes it possible to benefit from chaos engineering techniques without needing to construct experiments or run reliability tests.

Gremlin CTO Kolton Andrus said that approach eliminates much of the friction that previously limited adoption of chaos engineering best practices to improve the resiliency of IT environments.

Chaos engineering is rooted in a resiliency testing philosophy that posits applications should be able to keep functioning regardless of any service or infrastructure failure. Most IT organizations, however, don’t have the expertise required to build the tools required to test resiliency based on the “Chaos Monkey” principles—pioneered by a Netflix project by that name—that, for example, randomly turns services off to see if an application fails.

Each DevOps team will need to evaluate Gremlin’s recommendations before implementing them, but the overall goal is to provide IT teams with a better understanding of the level of risk attached to, for example, a misconfiguration of a server. To do so, the tool uses a scoring mechanism provided by Gremlin, noted Andrus.

The challenge is that as IT environments become more complex, the level of dependencies between applications and infrastructure continues to increase. It’s often impossible for DevOps teams to manually keep track of all the hidden dependencies that might exist when IT environments are continuously being updated.

In theory, of course, adopting cloud-native applications based on microservices should make IT environments more resilient as calls to application programming interfaces (APIs) are rerouted in the event of an outage. Nevertheless, there are almost always dependencies that get overlooked as these applications evolve, which creates a potential single point of failure.

Naturally, not every IT team is always comfortable with the concept of deliberately breaking things to better understand what issues might cause an outage. As organizations become more dependent on software to drive critical processes, however, a more proactive approach to ensure resiliency is required, added Andrus.

In fact, business leaders are becoming less tolerant of outages that result in lost revenue, so DevOps teams are being held to a higher level of accountability. DevOps teams may appreciate the value of a blameless culture that encourages everyone to learn from their mistakes, but IT leaders still get fired when there are too many outages, noted Andrus.

Less clear is the degree to which the rise of artificial intelligence might affect chaos engineering. At the very least, AI models should make it easier to assess levels of risk, but the cost of using those technologies to achieve that goal is substantial, especially when platforms that provide the same outcome already exist.

Regardless of approach, however, when it comes to resiliency, everyone agrees that the value of application uptime is now beyond measure.

August 30, 2023
VS Code for Mainframe: The Enterprise Developer Tipping Point

Optics matter. At a recent industry conference, a university lecturer reconfirmed what many in attendance already knew, “3270 [green screens] scare off the young generation of developers.” No matter how state-of-the-art the platform or mission-critical the applications, perceptions are formed largely based on optics, and green screens are not a good look.

Beyond optics, though, a new option offers the promise of a more normalized developer experience (DevX) and DevOps productivity across platforms, along with the staffing flexibility it brings. In short, Visual Studio Code (VS Code) is a big step toward breaking down the mainframe silo.

VS Code’s extensible, ‘any language, any platform’ design makes it the perfect tool for mainframe developers and with the free and open Code4z extension pack, VS Code is mainframe-ready. Developers are now using VS Code for mainframe AppDev just like their peers, taking advantage of the extraordinary ecosystem and the other many benefits that have made it a runaway success.

Per my previous article, Git for Mainframe: Back to the Future, combining VS Code with Git (i.e., editor and source interfaces) makes the experience complete.

So how does Code4z make VS Code mainframe-ready? Let’s explore the answers from a functional perspective.

Editing & Debugging

Mainframe is closely associated with core languages like COBOL, Assembler and JCL, which are often ignored in the world of modern tooling. With VS Code’s extensible framework, however, incorporating these languages is simple and seamless.

The Language Server Protocol (LSP), which was developed for VS Code, is now an open standard also used by the next generation of browser-based editors, providing a degree of future-proofing. LSP extensions provide a rich editing experience with a wide range of popular features including IntelliSense, which offers capabilities like syntax checking, verb formatting suggestions and variable name completion.

Code4z offers LSP extensions for COBOL and High Level Assembler (HLASM) while additional extensions are available for languages like JCL, REXX, Db2, and Easytrieve. The extensibility of this technology has already been used to extend syntax awareness to specific language dialects.

Similar to LSP, the Debug Adapter Protocol (DAP) provides a native interface and layer of abstraction for debug services. Code4z’s debugger extension facilitates the debugging of CICS and batch COBOL programs, an integral part of the end-to-end experience.

Access & Security

Mainframe applications, the lifeblood of many businesses, require rock-solid security throughout the entire SDLC. Code4z leverages Zowe, the award-winning open source framework that extends mainframe security and access controls to tools like VS Code. The Zowe Explorer extension, built on the Zowe CLI, provides access to data sets, jobs and USS files and developers also use it to issue TSO and MVS commands and to submit JCL. Zowe Explorer is packaged with Code4z as is a data editor for mainframe data sets including VSAM. This extension is built on Zowe Explorer, again leveraging the extraordinary extensibility. A Zowe Explorer for CICS is also available.

Similarly, developers can access a rich and growing set of conformant Zowe CLI extensions via the terminal and scripting.

Access to SCMs tools is also supported. In addition to VS Code’s native Git integration, there are extensions for Endevor, ChangeMan ZMF, and related tools.

DevOps Automation

Since my introductory article a few years ago, Z is for Zowe – the Open Path to Mainframe DevOps, Zowe has enjoyed wide adoption with 69% of surveyed sites either currently using or planning to use it within the year.

With mainframe teams now using Zowe to adopt modern shift-left practices, scripts for scanning, building, deploying and testing can be run manually by developers from VS Code, in addition to automatic triggers from code commits. This flexibility further empowers developers to own their code throughout the SDLC and beyond.

With VS Code, mainframe developers can be equipped with more self-service capabilities like Db2 schema provisioning to accelerate the testing process. With capabilities like these, developers can remain in the VS Code experience focused on editing rather than being distracted by low-value infrastructure activities, like requesting and waiting for DBA action.

Playing the Long Game

While VS Code is the dominant tool today—almost 3 times as popular as its closest competitor according to Stack Overflow with the lead growing—what is the long-term opportunity?

Demand for a modern, enterprise-aligned experience spans many domains, from increasing productivity to backfilling retiring staff. Elevating team morale and future-proofing vital mainframe assets ensure this adoption trend will only increase.

But tools change. Browser-based editors and in-the-cloud collaboration are gaining greater acceptance and the AI-assist excitement is palpable. Bringing mainframe developers onto an enterprise platform ensures mainframe teams will be able to benefit from the same advances as distributed peers. Enabling VS Code and its cloud equivalents demonstrates a commitment to the long-term success of these applications and by reducing the idea-to-production cycle, product owners become champions.

While not all mainframe developers will embrace VS Code, especially those close to retirement, the vast majority will, as we have experienced within our Broadcom engineering organization. The bottom line is that VS Code makes them even more productive, as illustrated in Confessions of a Career ISPF Programmer.

Many of today’s developers are proficient with multiple languages (i.e., polyglot) so picking up an additional language like COBOL becomes less daunting if everything else—editor, repo, DevOps processes, etc.—is familiar.

Getting Started

For a closer look at the many free and open Code4z extensions, check out the Developer Cockpit, which simulates a COBOL/Db2/CICS developer configuration. And read From VS Code to Mainframe Developer Cockpit for more detailed use cases.

Most importantly, try it! Code4z is available on the VS Code Marketplace.

And stay tuned for additional Code4z extensions in the pipeline. There’s never been a more exciting time to be a mainframe developer!

August 30, 2023
Convert COBOL Easily? Yes, Please!

I was reading Mike Vizard’s article about IBM previewing an AI tool to convert COBOL to Java and mentally cheered this move. No, seriously—massively cheered this move.

There are a lot of uses of large language models (LLM) right now, and many of them are not all that exciting, in my opinion. This one, though? This is huge. In the course of my career, I have been involved in three projects to convert COBOL and one to convert FORTRAN, with some projects that required interfacing with RPG III systems. In all of these cases, the costs were prohibitive. Two of the three COBOL projects never made it past estimates because the cost of resources who know both COBOL and Java were prohibitive, as is the complexity of most mainframe applications as they were modified over the years. The Fortran project never got past resourcing, and the RPG III interfaces were just that. We had to pull the data as it was presented by RPG III systems and only act on our copy, with a separate process used to update the core systems. Why? Because modifying the RPG was impractical, but we could be agile. It was … painful, to put it mildly.

What all of these projects shared was the lack of a skilled workforce. And what IBM is offering is an LLM that acts as that workforce. If you get the application into Java, there is a large pool of developers that can then troubleshoot it. While the environment issues will still exist and people skilled in tools like JCL are increasingly hard to find, that part can be rebuilt in the new environment. (One question I have about IBM’s project is, “Can it convert to Java for a Linux LPAR/VM?” I suspect that this iteration converts within z/OS, but it would be cool to see increased options in the future.)

Honestly, this trend will only increase. While IBM’s use of LLM is aimed at moving COBOL to Java, there are an incredible number of lines of Java out there to train on, so soon enough, there will be other conversions in and out of Java. And that’s all good news. We recreate the wheel a lot. The longer-term implication of AI-assisted conversion is that we won’t have to. Think about it: We could feed in a mathematical routine in Fortran and get the language of our choice out the other side.

Way back in the day, a collection of compilers named TopSpeed let you develop in any of several languages, even mixing and matching to create applications that used each language (and developer) for their strengths. This is the TopSpeed dream writ large. (Disclaimer: I was a huge TopSpeed fan. Bought their intro kit and wrote systems mixing C/Pascal/ASM without having to mess with things like parameter sequences. Yeah, I was cool.)

We have had too few developers for a very long time. Now, we are about to have a world where developers can focus on truly unique functionality, and the rest is generated based on the mass of implementations we already have. Think in terms of sorting algorithms—99% of the sorting problems out there were mastered decades ago, and yet we frequently see people re-implementing, even today. Now we can tell a system, “Generate a tree sort,” and just make calls. Every time we implement a new language, this type of thing has to be re-implemented, and often it is re-implemented because we don’t know about the available solutions. In the future, we will have solutions at our fingertips.

And as with the other benefits of automation, we’ll do less busy work and add more business value. While there will be some pain, I think in the long term that is a huge benefit. And you’ll still be there, making things go zoom.

August 30, 2023
Apica Acquires LOGIQ.AI to Unify Observability

Apica, a provider of a synthetic monitoring and observability platform, has just raised an additional $10 million in funding and has agreed to acquire LOGIQ.AI. LOGIQ.AI is a provider of a data fabric infused with machine learning algorithms.

Jason Haworth, chief product officer for Apica, said the data fabric developed by LOGIQ.AI will be incorporated into the Apica Ascent platform in the third quarter. The LOGIQ.AI addition will apply artificial intelligence (AI) in a way that both optimizes and enriches data collected by the company’s observability platform.

At the core of the Apica Ascent platform is an indexing engine built on top of a Kubernetes platform that aggregates data such as logs, traces and network packets from multiple sources. As part of that process, the platform reduces storage costs by trimming excess data that can be stored in a data lake it provides or a third-party data lake a customer prefers.

At the same time, Apica also enables DevOps teams to both normalize data collected from multiple sources and enrich it to add additional context, said Haworth.

The overall goal is to provide a unified view of all IT data to accelerate faster root cause analysis and reduce the total cost of collecting the data required, he added.

Apica is making a case for an observability platform that goes beyond collecting data from DevOps platforms. As IT environments become more complex, IT teams need to be able to correlate data collected from a wide range of data sources to accurately determine the root cause of an issue, said Haworth.

The challenge is managing the massive amount of data that needs to be continuously collected, stored and analyzed to achieve that goal, he added. The acquisition of LOGIQ.AI will enable Apica to embed a data fabric that employs machine learning algorithms and other data science techniques to make it feasible to observe complex IT environments at scale, noted Haworth.

It’s still early days as far as observability is concerned, but it’s clear that IT teams need to move beyond monitoring a pre-determined set of metrics to manage complex and dynamic application environments. The rate of change to application environments as updates are made requires an ability to query data to determine the root cause of issues that could be caused by any number of dependencies.

The issue is that even when IT teams have access to an observability platform, they may not have the knowledge and expertise required to craft the queries needed to determine the root cause of an issue. Machine learning algorithms will play a major role in enabling IT teams to proactively discover issues that they can investigate using the suggested queries.

It’s only a matter of time before most IT organizations unify observability data within a single platform. That will make it simpler for them to collaborate across teams and should result in fewer disruptions as IT issues are discovered more quickly. After all, the best kind of IT incident is the one that never happened or that was too trivial for anyone outside of IT to even notice.

August 29, 2023
Styra Makes Source Code Available for Enterprise Edition of OPA

Styra is now making the enterprise edition of the Open Policy Agent (OPA) available under a source license that enables IT teams to modify or change the underlying code as they see fit when integrating it with other applications.

Enterprise OPA is a drop-in replacement for OPA, agent software for implementing policy-as-code that is already available under an open source license. Styra’s enterprise edition provides access to a version that provides better performance, native integrations for a suite of data sources and additional capabilities such as policy decision analysis.

Previously, Enterprise OPA was only available in either a binary or container format. While these formats enable Enterprise OPA to be widely used, they limited it from being built directly into applications for performance optimization because no access was provided to source code.

Chris Hendrix, director of product management for Styra, said the source license also provides organizations with an assurance that they will be able to continue to modify Enterprise OPA in much the same way as they modify the open source edition.

It’s not clear how many DevOps teams are managing policy-as-code within their pipelines, but as application security regulations become more stringent, it’s only a matter of time before most of them do so in one way or another. Styra’s approach relies on a Rego programming language that some DevOps teams have found challenging, however. To make it simpler, the company is working on a low-code approach that would help make Enterprise OPA more accessible, said Hendrix.

In addition to encouraging more DevOps teams to use Enterprise OPA under a source license, Syra is clearly moving to make it more attractive to replace existing instances of OPA with a more robust implementation licensed from the company. No one knows how many instances of the open source edition of OPA advanced under the auspices of the Cloud Native Computing Foundation (CNCF) are running in production environments. But as those applications become more complex, the need for deeper levels of integration enabled by a faster version of OPA becomes more apparent.

OPA enables developers to programmatically embed compliance controls without waiting for a compliance team to make them available. That ultimately makes it simpler to ensure that the proper controls have been implemented before an application is deployed in an on-premises IT environment. That capability also makes it easier to document which controls have been applied to make any audit less time-consuming.

Shifting responsibility for compliance controls to the left also enables DevOps teams to better understand which controls address specific regulatory mandates incorporated into most applications versus which controls might be specific to a specific regulation. Armed with those insights, it then becomes simpler to incorporate policy-as-code within a DevSecOps workflow.

Hopefully, more applications running in production environments will soon be secure by default. Naturally, one of the first places to start is by making sure the compliance controls needed to achieve a baseline level of security have been pervasively implemented.

August 28, 2023