devopsy.kinsta.cloud

Category: DevOps in the Cloud

SolarWinds Adds Service Management Platform to Portfolio

SolarWinds has added a service management capability to its portfolio as part of an effort to make it simpler for IT teams to align support services with specific functions.

At the same time, SolarWinds has updated its SQL Sentry observability tool for databases and DataOps workflows to add query-level wait statistics along with a revamped web portal and dashboard for tracking the overall health of a database environment.

SolarWinds Enterprise Service Management (ESM) makes it possible for IT teams to manage IT services based on the ticketing systems, knowledge bases, service portals and request catalogs available for specific departments such as marketing or human resources.

Kevin Kline, senior staff technical marketing manager for SolarWinds, said that as IT management continues to evolve, there is a greater need to streamline and optimize workflows based on the insight from observability platforms. As a result, there is now a need to apply observability more broadly to include database platforms alongside DevOps and other IT workflows, he noted.

The overall goal is to eliminate the blame game that results when IT platforms are managed in a more siloed fashion, added Kline.

Most IT teams are still relying on monitoring tools to track a set of predefined metrics in the hope that an alert will enable them to proactively address an issue before it leads to a major disruption. However, as IT environments become more complex, there is a growing need for observability platforms so IT teams can launch queries and surface issues that predefined metrics aren’t likely to identify. Initially, observability platforms were embraced by DevOps teams, but it’s rapidly becoming apparent that this capability needs to be applied to every IT process. In effect, organizations are on the cusp of democratizing observability, noted Kline.

Achieving that goal will also become simpler thanks to the rise of machine learning algorithms capable of predicting issues and generative artificial intelligence (AI) tools that will make it simpler to craft queries.

In the meantime, there is already no shortage of observability platforms, but the SolarWinds approach is different in that, in addition to DevOps workflows, the company has integrated its platform with multiple database platforms to provide a more comprehensive approach, said Kline.

It’s not clear just yet how observability platforms might impact the way IT teams are structured, but the need for specialists such as database administrators (DBAs) isn’t likely to go away anytime soon, noted Kline. Instead, the level of collaboration across various IT teams should substantially improve.

In the meantime, IT teams might be well-advised to define an observability strategy today before multiple teams start implementing approaches that might be optimized for a relatively narrow range of use cases. After all, trying to reconcile insights generated by multiple observability platforms may very well defeat the purpose of investing in observability in the first place.

September 25, 2023
Tray.io Automation Framework Leverages AI to Build Integrations

Tray.io today added an automation framework to an integrated platform-as-a-service (iPaaS) service that leverages an existing Merlin artificial intelligence (AI) engine to identify the most efficient way to automate a workflow.

The Tray Universal Automation Cloud service can be invoked via three distinct application programming interfaces (APIs) to programmatically create those workflows using a Tray Code low-code or Tray Build no-code tool or via a Tray Chat natural language interface. A Connectivity API provides access to more than 600 connectors, while a Trigger API enables developers to drive event-driven processes. An Authentication API provides a method for managing authorizations. The platform also provides access to testing, error handling and debugging tools.

Tray.io CEO Richard Waldron said that approach provides IT teams with a composable framework for building integrations that leverages a set of large language models (LLMs) that the company has embedded within its Merlin AI engine to enable IT teams to optimize workflows.

In addition, the underlying platform takes advantage of Kubernetes, Docker containers and serverless computing frameworks to dynamically scale as required, he added. In contrast, rival legacy iPaaS environments are not as elastic, which enables Tray.io to base its pricing on actual usage of a modern IT platform, noted Waldron.

The overall goal is to provide a single platform that any type of developer, including business professionals, can invoke to automate workflows, said Waldron.

As the overall codebase that DevOps teams manage continues to increase, the number of required integrations between applications and processes continues to increase exponentially. Rather than relying on lower-level code to create those integrations, an iPaaS provides the means to automate and simplify building integrations and enable developers to spend more time writing the business logic that will drive a new application. Historically, most of the integrations required to drive, for example, a digital business transformation initiative required a significant amount of low-level code to be developed and maintained.

Tray.io is now making a case for an iPaaS that takes advantage of AI to more easily invoke generative AI capabilities to optimally create those integrations based on workflow attributes.

Each organization will need to determine how heavily to rely on AI platforms to build integrations, but these advances—alongside generative AI tools that developers have already embraced to build software—are making developers more productive than ever. That accelerated pace of application development will require DevOps teams to automate as many processes as possible. One option for achieving that goal is for organizations to rely more on an iPaaS to enable developers to build integrations in a way that doesn’t necessarily always have to flow through a DevOps pipeline.

One way or another, thanks to the rise of AI, the way software is built is about to change drastically. The challenge and the opportunity now is to determine how best to reengineer workflows in a way that best aligns with the goals of the organization.

September 25, 2023
Five Great DevOps Job Opportunities

devopsy.kinsta.cloud is now providing a weekly DevOps jobs report through which opportunities for DevOps professionals will be highlighted to better serve our audience.

Our goal in these challenging economic times is to make it easier for DevOps professionals to advance their careers.

Of course, the pool of available DevOps talent is still relatively constrained, so when one DevOps professional takes on a new role it tends to create an opportunity for others.

The five job postings shared this week are selected based on the company looking to hire, the vertical industry segment and, naturally, the pay scale being offered.

We’re also committed to providing additional insights into the state of the DevOps job market. In the meantime, for your consideration:

Dice.com

R&T Deposit Solutions
New York, New York
DevOps Engineer
$145,000 to $175,000

Indeed.com

Caterpillar
Chicago, Illinois
DevOps Engineering Manager
$141,000 to $178,000

SimplyHired.com

Takeda Pharmaceutical
Boston
Machine Learning Infrastructure/Platform Engineer
$130,000 to $186,000

LinkedIn

M&T Bank
Wilmington, Delaware
DevOps Manager
$123,000 to $206,000

CareerBuilder.com

Fidelity Investments
Westlake, Texas
DevOps Engineer
$120,000 to $160,000

September 25, 2023
There’s No Value in Observability Bloat. Let’s Focus on the Essentials

The observability market paired with cloud security has been predicted to reach $70B by 2026, but that market prediction is a myth, and I’ll tell you why.

If observability was a grand buffet and every company needed a feast of features and capabilities of this proportion, maybe this prediction would be valid. But most companies don’t need an over-the-top feast. Legacy APM and observability solutions are serving up an all-you-can-eat buffet when the reality is that most companies only need the essential ingredients to satisfy their observability strategy. And the cost for that can be much more modest. The $70B observability market prediction is, therefore, suspect, considering the true needs of most organizations.

Complex cloud ecosystems, the rise of applications built on microservices and Kubernetes, and the overall scale and complexity of applications have led to exponentially more data being created in the pursuit of observability. Storing and processing all that data is expensive, and as the volume of data balloons, so does the cost.

And while the complexity of business-critical systems and the volume of data have scaled, so have the overall cost of those legacy tools—in direct proportion. Partially, this is driven by the features and instrumentation added that only contribute to the bloat and valueless telemetry.

Customers aren’t seeing value in any of this bloat, and for the most part, all that’s been accomplished is increased complexity in deploying and operating the system, with a commensurate lengthening of time to value. Plus, pricing models haven’t significantly changed despite the advancements in the market. The fact is, organizations can’t really say that they have gotten more insight into the overall performance of their mission-critical systems. What they are saying, however, is that these tools are becoming overwhelming to manage and overwhelmingly expensive, and developer productivity is not increasing.

What organizations need and want are real-time insights into the health and performance of their cloud infrastructure and applications–specifically the confluence of logs, metrics and traces within the context of modern observability. There is little tolerance for software bugs, slow web experiences, crashed apps and other service interruptions, and the responsibility lies on developers, engineers and ITOps teams to quickly resolve production issues before they impact customer experience.

There is a call for observability, but rather than an explosion of features and noise, what’s being called for is a narrowing of focus.

Observability Has an Efficiency Problem

Telemetry data gathered from the distributed components of modern cloud architectures needs to be centralized and correlated for engineers to gain a complete picture of their environments.

Engineers need a solution with critical capabilities such as dashboarding, querying and alerting, and AI-based analysis and response, and they need the operation and management of the solution to be streamlined.

What’s important for them to know is that it’s not necessary to spend more to ensure peak performance and visibility as their environmental complexity grows.

Legacy players experiencing some maturity in the market may have conditioned them to believe this is true—they have set a standard for higher costs correlating with incremental amounts of data, no matter the quality of that data. Companies are being fed the lie that all this data may be relevant and they’ll be doing themselves a disservice by neglecting even one byte of it.

Data Does Not Inherently Equal Insights

No doubt, more data is being generated, but most of it is not relevant or valuable to an organization. Observability can be optimized to bring greater value to customers, and that’s where the market is headed.

Call it “essential observability.” It’s a disruptive vision to propose a re-architected approach to observability, but what engineers need is a new approach making it easier to surface insights from their telemetry data while deprioritizing low-value data. Costs can be reduced by consuming only the data that enables teams to maintain performance and drive smart business decisions.

Essential observability is cutting out the fluff and only focusing on—and paying for—the impactful data. It’s about arming engineers with meaningful insights so they can quickly resolve production issues and business disruptions. Organizational performance increases without inflating data complexity and resource consumption.

The Essential Observability Revolution

Observability tools have evolved beyond the model of linear price increases as complexity and data scales. For companies (especially growth-stage organizations) that want a partner to help them make more informed decisions and keep their costs more reasonable (i.e., stop recklessly spending for useless data), they need to focus on tools offering them the essentials.

Solutions are available, offering targeted features and automated capabilities out of the box, delivering the insights needed to understand and fine-tune data. These solutions provide continuous support, reduce complexity and accelerate time to value.

The next revolution in observability isn’t features and functionality but providing insights to make data-driven decisions that keep essential observability cost-effective while not compromising the value derived.

September 22, 2023
Oracle Previews Latest Java 21 Innovations

Oracle today made generally available a Java 21 edition and provided early access to more than 15 proposed enhancements to the Java development kit (JDK).

Those enhancements were announced at an Oracle CloudWorld 2023 conference, including support for virtual threads to improve overall throughput and adding support for a vector application programming interface (API) that makes it simpler to build Java applications that involve artificial intelligence (AI) models.

Georges Saab, senior vice president of Oracle Java Platform and chair of the OpenJDK governing board, said the goal is to streamline access to the latest innovations that will be added to future versions of Java.

Specifically, via Project Amber, JDK 21 delivers improvements such as support for string Templates, record Patterns, pattern Matching for switch, unnamed Patterns and variables and unnamed classes and instance main methods.

In addition to the Vector API, Project Panama adds a Foreign Function & Memory API. Project Loom, in addition to virtual threads, adds scoped values and structured concurrency support.

There is also a more efficient garbage collection mechanism being proposed.

Oracle is committing to offering long-term support for Java 21 for at least eight years to encourage organizations to start working with this version of Java today. Oracle is also making available a Java Playground, an online sandbox on the Oracle Cloud Infrastructure (OCI) service where developers can create Java code using this edition without needing a local runtime or integrated development environment (IDE).

In general, Java has been experiencing a renaissance after more organizations began contributing to projects once the core platform became available under an open source OpenJDK license. In fact, the Java community is now adopting many concepts pioneered in other programming languages. That enables organizations to, for example, drive digital business transformation initiatives that depend on the modernization of legacy applications. As part of those efforts, the rate at which Java applications are being deployed in the cloud has substantially increased.

Despite the number of alternative programming languages, many enterprise IT organizations remain committed to Java. Rather than opting to retrain developers, it’s simply been more cost-effective to enable developers to continue to build applications using a programming language they already know.

Many organizations are currently in the process of shifting to Java 17. It’s not clear how many organizations will skip Java 17 and jump to Java 21 from previous versions of Java, but it’s clear Oracle is committed to making the lag time between moving from previous versions of Java to Java 21 much shorter.

Most organizations will find themselves continuing to deploy Java applications everywhere from the network edge to the cloud. These applications will be written in a programming language that has persevered for more than two decades and continues to evolve and mature for the next era of application development.

September 19, 2023
Buildkite Acquires Packagecloud to Streamline DevOps Workflows

Buildkite Pty Ltd today revealed it has acquired Packagecloud as part of an effort to tightly integrate package management tools with its continuous integration/continuous delivery (CI/CD) platform.

Buildkite CEO Keith Pitt said the goal is to reduce the level of friction that exists between providers of third-party package management tools and CI/CD platforms that DevOps teams are today required to address themselves.

There is no shortage of package management tools used to configure, install and update software, but the level of integration between these tools and CI/CD platforms has not historically been very high. The acquisition of Packagecloud will result in a package manager becoming a first-class citizen within the context of a CI/CD platform, said Pitt.

Buildkite will maintain the existing support Packagecloud provides for platforms such as Jenkins, but the acquisition will streamline DevOps workflows and improve overall developer productivity, he added.

In the months ahead, Buildkite will also add a range of tools to better enable DevOps teams to achieve that goal, said Pitt. In addition, the company plans to add tools to streamline the management of compliance processes as regulations pertaining to application development are becoming more stringent, he added.

It’s not clear how much IT organizations are reevaluating their CI/CD platforms to improve developer productivity, but platform engineering as a methodology to centralize the management of DevOps workflows is gaining traction. Those efforts promise to reduce the total cost of implementing DevOps at scale and improve developer productivity by making the scaffolding required to build applications more readily accessible. Today, too many developers are spending time managing developer environments rather than writing actual code.

The challenge, of course, is convincing developers—who tend to jealously guard their tool prerogatives—to embrace a platform engineering methodology and cede control over the development environment to a centralized team.

One way or another, developer productivity needs to improve as more organizations encounter economic headwinds. Developers are the most expensive resource any organization has, so improving the rate at which high-quality code is created will always be imperative, noted Pitt.

Today, organizations depend on software to drive revenue. Investors are assessing senior leaders within those organizations on their ability to deliver quality software in a timely fashion. Each new application and subsequent software update now has a direct impact on the profitability of the organization. As such, tolerance for legacy approaches to building software that require more time and effort is dropping.

The issue, as always, is that switching from one software development platform to another is an expensive proposition, so many organizations may find themselves supporting a mix of legacy and modern platforms. The challenge becomes defining the pace at which that migration occurs as new applications are created and legacy applications are continuously updated.

September 19, 2023
DevOps is Making Gains on Mainframe Platforms

A survey of 800 executive and technical professionals conducted by BMC found close to two-thirds (62%) are employing DevOps workflows to build and deploy applications.

In addition, the survey also finds that just under a third of respondents (32%) are applying artificial intelligence to IT operations (AIOps) across all platforms.

John McKenny, senior vice president and general manager of intelligent Z optimization and transformation at BMC, said the survey shows that organizations continue to invest in mainframe applications alongside distributed computing environments.

In fact, 61% of respondents noted that data volume on mainframes increased in the last year, with 56% reporting an increase in the number of databases running. Well over a third of respondents (35%) also reported increased interest in taking advantage of cloud technologies to handle, for example, storage and data protection.

Overall, the survey found that 94% of respondents have a positive perception of the mainframe, with nearly two-thirds (63%) reporting increased mainframe investments over the past year.

Compliance and security lead the list of priorities for mainframe organizations, with 61% of respondents citing these as top issues, followed by cost optimization (49%), enhancing automation (41%) and data recovery (40%).

In general, organizations with mainframes are adopting modern application development tools such as VS Code to attract a younger generation of developers. It’s critical to continue to attract developers to mainframe platforms that continue to run the most mission-critical applications in the enterprise, said McKenny. Many of those applications are being modernized using DevOps tools and practices rather than being moved to another platform, he added.

He added that the appetite for using generative artificial intelligence (AI) to write code for mainframe applications is less clear because of intellectual property concerns. Most organizations are not going to want to expose their mainframe code to a general-purpose large language model (LLM) that drives a public generative AI platform, but there will be a role for private LLMs that are trained using code that has been validated, noted McKenny.

Regardless of how generative AI evolves, there’s still much work to be done to further the adoption of DevOps best practices in mainframe environments. Each organization will determine how stringently to apply those practices based on the frequency at which applications running on mainframes need to be updated. In most cases, organizations with mainframes will use a mix of waterfall and DevOps-based approaches to building and deploying applications for many years. The decision as to which approach to employ will, naturally, have as much to do with culture as it does the rate at which the application needs to be built and deployed to drive, for example, a digital business transformation initiative.

The days when mainframes were managed in isolation from the rest of a distributed computing environment are slowly coming to an end. A mainframe is, arguably, just another node for deploying applications. The challenge and the opportunity now is getting the right DevOps tools in the hands of the IT team managing heterogeneous IT environments, including mainframes.

September 15, 2023
There’s no One-Size-Fits-All Data Mobility Strategy

Whether you’re a Fortune 500 enterprise or a fledgling startup, there’s one thing every business can expect to confront: Change. This is resoundingly true when it comes to data management. Even greenfield operations will, sooner rather than later, need to change how and where they leverage their data. After all, every business—including those launched with the most cutting-edge technology—is built on a series of choices and compromises. As technologies and business priorities change, so should your data mobility strategy.

In a world of ever-changing variables, what’s the right data mobility strategy? There is no one-size-fits-all solution. It does, however, help to start with a solid foundation on which to build your strategy.

With the right platform that allows for flexibility and change, it’s time to build the structural framework for your data mobility strategy. When creating the framework for your data mobility strategy, there are a few considerations to keep in mind, including your desired use of on-premise storage, your hybrid cloud strategy and “supercomputing” capabilities, and the storage area network (SAN) needed based on your organization’s needs. Let’s review what they might mean for you:

Developing a Hybrid Cloud Strategy

The reality is that the hybrid cloud is now achievable for most organizations. According to Flexera’s 2023 State of the Cloud Report, 72% of organizations have embraced a hybrid cloud strategy. Your data mobility strategy should reflect that.

With a proper hybrid cloud environment, the user experience should be seamless regardless of where your data exists, whether that’s on-premise, in a private cloud, or in the public cloud. As the potential for more dynamic resource allocation grows, your data mobility strategy inherently becomes more critical. Data mobility, in effect, now equates to computing mobility.

Leveraging the “Supercomputer” in the Cloud

Data mobility becomes even more imperative as entities gain more opportunities to take advantage of “supercomputing” capabilities.

The computing power that exists in the cloud now is tremendous, and thanks to modern dynamic cloud pricing, even smaller organizations can tap into it. According to one estimate, the cloud-based high-performance computing market should reach $55 billion by 2030, growing at a CAGR of 7% from 2023 to 2030.

So for example, if you’re a FinTech startup, you could pay by the minute to leverage high-performance computing capabilities to test your trading algorithm. Previously, these powerful capabilities were only available to the biggest customers — think NASA — and came with astronomical costs.

On-Premise Solutions Aren’t Going Away

Even as cloud services and capabilities become more accessible and more valuable, on-premise operations aren’t going to disappear. Customers have all kinds of reasons for keeping data on-premise, whether it’s for compliance purposes, costs or the desire to leverage existing infrastructure. Certain workloads, meanwhile, simply don’t make as much sense on the cloud.

The reality is that even the most compelling alternatives couldn’t eliminate on-premise operations, because business ecosystems and industries evolve organically. Think about it: Many businesses still use landlines, and workers still use desktop computers. Nearly half a century after marketers first coined the term the “paperless office,” it took a full-blown pandemic to push our collaboration tools to the point where we are finally, basically, paperless.

Fiber Channel and iSCSI

When it comes to data mobility, how does an organization decide whether to adopt fiber, iSCSI or both? As organizations manage workloads that require low latency and other networking capabilities, they’re going to be investing heavily in this area.

Traditionally, a significant amount of data management is fiber-channel-based. With iSCSI, an organization doesn’t need to invest in separate physical infrastructure, such as switches or cables. Moreover, iSCSI has become more attractive to entities moving workloads to the cloud. A whole range of products, from Pure Storage, NetApp and even cloud platforms like Microsoft Azure, offer iSCSI-based storage in the cloud. Your data mobility strategy should consider how to move from fiber channel to iSCSI and back again.

Change is the Only Constant

Ultimately, no one can predict all of the major technological innovations that will impact their business down the road. To stay nimble, you need a data mobility strategy that you can execute with the team you have — whether their primary expertise is in storage, networking or elsewhere. This can be especially challenging in large organizations, with teams that have to meticulously plan their projects and stay accountable for their budgets. Now more than ever, it’s important to have a solid foundation on which to build a strategy that fits your needs.

September 15, 2023
Google De-Recruits 100s of Recruiters ¦ ARM Valued at $45½B in IPO

Welcome to The Long View—where we peruse the news of the week and strip it to the essentials. Let’s work out what really matters.

This week: Google fires hundreds of recruiters, and ARM gets a sky-high valuation. (more…)

September 14, 2023
CloudBees Unfurls Dual CI/CD Strategy at DevOps World Event

During a DevOps World event, CloudBees took the wraps off the first major update to the open source Jenkins continuous integration/continuous delivery (CI/CD) platform made in the past several years.

At the same time, the company also unveiled a proprietary DevSecOps platform based on Kubernetes that is optimized for building and deploying cloud-native applications based on portable Tekton pipelines.

Shawn Ahmed, chief product officer for CloudBees, said the updates to Jenkins make it possible to horizontally scale Jenkins controllers while at the same time ensuring high availability. The latest update also provides a workspace caching mechanism to simplify sharing them across multiple jobs in a way that significantly reduces the amount of compute and storage resources consumed, he added.

CloudBees also added a Baseline capability to the Jenkins repository to improve startup times without causing build storms.
Collectively, those capabilities will increase the number of jobs that can be simultaneously executed to improve overall productivity, said Ahmed.

CloudBees has also added a Pipeline Explorer tool in the cloud-based edition of Jenkins that enables DevOps teams to visually debug and troubleshoot pipeline jobs. That tool will also provide the foundations for self-healing and artificial intelligence (AI) capabilities that CloudBees plans to add to the Jenkins platform later this year and into 2024, added Ahmed. For example, AI tools will surface suggestions on how to fix a problem in a pipeline, he noted.

The DevSecOps platform, meanwhile, leverages Tekton pipelines to provide DevOps teams with more granular control over how pipelines can be orchestrated, said Ahmed. It also provides access to GitHub Actions-style domain-specific language (DSL), support for feature flagging, security, compliance, pipeline orchestration capabilities, analytics and value stream management (VSM) tools that can be deployed by an internal IT team or accessed via a single or multi-tenant managed cloud service.

Those built-in capabilities will make it simpler for platform engineering teams to centrally manage building and deployment of cloud-native applications by enabling developers to self-service their own engineering requirements via an internal developer platform (IDP), said Ahmed. The overall goal is to provide a more integrated approach that breaks down software engineering silos that exist in many existing DevOps workflows, said Ahmed.

Each DevOps team will need to decide for themselves to what degree to move forward using Jenkins to build and deploy applications versus opting for a more modern alternative in the form of the DevSecOps platform. CloudBees, however, is committed to providing plug-ins and integrations across both platforms to enable DevOps teams to protect their investment in existing tooling, noted Ahmed.

The status of Jenkins X, an open source CI/CD platform based on Kubernetes that CloudBees previously donated to the Cloud Delivery Foundation (CDF), is not immediately clear. Regardless of the approach, however, there is a clear opportunity to revamp DevOps workflows by upgrading Jenkins or switching to a more modern platform such as DevSecOps. The challenge, as always, is determining the pace at which any such transition might be made.

September 14, 2023