devopsy.kinsta.cloud

Category: Features

Cribl Simplifies Telemetry Data Analytics

Cribl today announced general availability of a search capability that makes it simpler to query observability data where it resides without having to first collect and centrally store it.

Nick Heudecker, senior director for market strategy for Cribl, said Cribl Search makes it possible to analyze telemetry data at its point of origin or when it is stored in a low-cost object storage platform to minimize costs.

As the volume of telemetry data being collected continues to grow, DevOps teams are struggling with ingesting it all. Cribl takes advantage of a platform it created to stream and normalize the collection of telemetry data. Cribl Search adds the ability to also query that data at the endpoint where it resides, said Heudecker.

While observability has always been a core tenet of DevOps, achieving it has been a major challenge. In addition to deploying and managing agent software to instrument IT environments, there’s a significant amount of time and effort required to set up the pipelines necessary to transfer data to a platform for storage and analysis. The more data is collected, the greater storage costs become. Cribl is making a case for automating those tasks using a tier of software that abstracts data into a format that can be more easily collected and analyzed.

In general, observability is emerging as the next logical evolution of continuous monitoring. Existing monitoring tools enable DevOps teams to track specific pre-defined metrics. Observability platforms are intended to make it easier to query telemetry data to discover the root cause of an issue that is not being tracked by a set of pre-defined metrics.

It’s not clear yet whether IT organizations perceive observability to be an evolution of monitoring or a distinctly separate but complementary capability, but it’s apparent IT teams need a lot more context than they have historically been able to attain using legacy monitoring tools. Each platform used within an enterprise typically comes with its own monitoring tools that are used by the IT teams tasked with managing it. Whenever there is an issue, it’s not uncommon for DevOps teams to spend hours correlating data from multiple tools to ascertain the root cause of a problem. Observability platforms promise to reduce that time and effort by applying analytics to data collected from multiple platforms. The challenge is finding a way to efficiently collect all that data, which in the past has often required a data engineer to build a pipeline spanning multiple sources of telemetry data.

One way or another, observability platforms should make it easier for DevOps teams to manage IT issues more proactively. The goal, as always, is to identify issues before they cause a major disruption. In that sense, it may be difficult to prove the ROI of deploying an observability platform. However, the best IT incident is always going to be the one that never happened or, at the very least, never escalated to a level of noticeable disruption.

November 9, 2022
VMware Extends Equinix Alliance to Drive Managed Hosting Service

VMware today announced it has extended its alliance with Equinix to launch a managed distributed cloud service that runs instances of its cloud computing platform in hosted IT environments.

Announced at the VMware Explore 2022 Europe conference, the VMware Cloud on Equinix Metal service takes advantage of more than 240 Equinix data centers in 71 major metro areas around the world to deploy instances of the VMware Cloud platform running atop the company’s virtual machine software.

Narayan Bharadwaj, vice president of cloud solutions at VMware, said this VMware service extends the scope and reach of the company’s multi-cloud computing strategy beyond the scope of the major hyperscalers and traditional on-premises IT environments.

The goal is to provide a managed IT environment that provides an alternative for deploying latency-sensitive applications closer to the network edge where data can be processed and analyzed in near-real-time, he noted. In addition, Equinix’s hosted IT environment also makes it possible for IT teams to more easily comply with requirements in highly regulated industries.

It’s not clear what percentage of applications today are running in managed hosted environments versus a public cloud or local data center. But as more organizations decide not to operate data centers themselves, many of them are discovering managed hosting services are preferable to the cloud for reasons spanning everything from application performance and cost and security to compliance.

In some cases, organizations are determining that certain classes of longer-running applications initially deployed in the cloud are actually less expensive to run in an on-premises IT environment, noted Bharadwaj. A managed hosting option often provides the flexibility benefits of the cloud at a lower cost for those applications, he added.

As cloud computing evolves, it’s become apparent that IT is becoming more distributed than ever. Many organizations are finding they now have a mix of applications running on public clouds, in co-location facilities and traditional on-premises IT environments. The challenge is that with each new platform employed to run applications, the total cost of managing IT increases. Regardless of any potential infrastructure cost savings, each platform typically requires a set of dedicated specialists to master the frameworks need to manage it. In fact, the single biggest IT expense remains the cost of labor. VMware is making a case for a familiar IT framework that can be deployed anywhere to rein in those costs.

DevOps, of course, was created to provide a new approach to automating the management of IT at scale. However, DevOps continues to be employed unevenly, largely because there are not enough IT professionals with the programming skills required to successfully manage IT-as-code. Managed services are gaining traction because, in most cases, they shift the responsibility for managing DevOps workflows onto a vendor such as VMware.

It remains to be seen just how automated the management of IT will become in the months and years ahead. However, it’s apparent that, in the era of the cloud, more organizations are opting to rely on third parties to manage infrastructure so they can devote more resources to building and deploying applications.

November 8, 2022
OutSystems Acquires Ionic to Further Automate App Development

OutSystems this week revealed it has acquired Ionic, a provider of an open source framework that provides developers with a higher level of abstraction for building and deploying mobile and desktop applications.

OutSystems CTO Patrick Jean said Ionic will accelerate the company’s ongoing efforts to extend the reach of its platform for building low-code applications without developers having to worry about, for example, the screen size of the device an application is deployed on it because the framework automatically right-sizes it. Ionic, in effect, will provide an extension to the platform that OutSystems already uses to automate the building and deployment of low-code applications, he added.

In addition to its namesake user interface (UI) framework for building mobile applications that can run on multiple platforms, the Ionic portfolio also includes a tool for natively deploying web applications, dubbed Capacitor, and a toolchain for building web component-based design systems, dubbed Stencil. Collectively, these tools are used today by more than five million developers.
OutSystems plans to extend the capabilities of these tools to make it simpler to deploy applications on both mobile devices and desktop systems, said Jean. The overall goal is to reduce the cognitive load that developers currently carry when they need to build and deploy applications that run on multiple platforms, he added.

That capability will prove critical as more individuals beyond professional developers become involved in building, deploying and updating applications, noted Jean. In fact, many of those developers are likely to be business people that only develop applications on a part-time basis, he added.

The percentage of applications being built using low-code tools has increased dramatically. Professional developers have embraced these tools and platforms alongside so-called citizen developers that are using them to build applications, as well. As the backlog of applications that needs to be built steadily increases, low-code platforms have played a crucial role in enabling developers to keep pace with the demand for new applications.

Less clear is the impact these low-code applications are having on DevOps workflows. On one hand, there are clearly more applications that need to be deployed than ever. At the same time, many of the DevOps and data modeling processes that once required, for example, a dedicated continuous integration/continuous delivery (CI/CD) platform are being embedded in the platforms provided by the vendors that create low-code tools.

Regardless of how low-code applications are deployed, the one thing that is certain is the pace of digital business transformation is being greatly accelerated by these tools and platforms. DevOps teams will soon need to put guardrails in place that enable developers with a wide range of experience and skillsets to securely deploy and update applications.

On the plus side, DevOps teams should also expect it to become easier to automate the delivery of low-code applications at levels of unprecedented scale. In fact, the issue going forward may not be whether IT can keep up with the needs of the business but whether the business can keep pace with the rate of innovation now being enabled.

November 8, 2022
SRE Survey Reveals Major Technical and Cultural Challenges

Catchpoint, in partnership with Blameless, today published an annual survey of 559 site reliability engineers (SREs) that found 59% of respondents didn’t view tool sprawl to be a major concern. Another 40% said tool sprawl is a moderate (33%) to serious problem (8%).

More than half of respondents said they build as much as 30% of the tools they use themselves, with more than half (54%) noting their organization tracks three or more different types of telemetry data to drive observability. The top three types of telemetry data collected are infrastructure (62%), application (58%) and network monitoring (55%) and, on average, the survey suggested the median toil level of SREs dropped 5% year-over-year.

Just under half of respondents (46%) also noted they saw no or low value from artificial intelligence for IT operations (AIOps) compared to 30% that have seen moderate to high value. Another 25% are still unsure.

Finally, the survey concluded that organizations that have adopted a “just culture”—where organizations ensure that incident reporting is not met with punitive measures—are 500% more likely to be Elite performers in terms of the DevOps Research and Assessment (DORA) metrics defined by Google.

Leo Vasilou, director of product marketing for Catchpoint, said that the survey results provided some insights into the challenges SREs face as they strike a balance between trying to achieve DevOps goals and the fact that hiring has become more constrained during an economic downturn.

That stress, however, has less to do with managing toolchains than it does ensuring that applications driving critical business processes are consistently available, he noted.

SREs have emerged as a distinct DevOps function in recent years, focused almost exclusively on programmatically managing the dependencies that exist between a wide range of applications and the underlying infrastructure upon which they depend. Optimizing applications in the cloud-native era has become more challenging because the number of dependencies that exist between the microservices that make up a modern distributed computing environment continues to exponentially increase. In many cases, applications are continually added to IT environments without a commensurate increase in the overall size of the SRE team.

Regardless of the economic climate, hiring and retaining SREs remains a challenge given the demand for IT professionals with those skills. Of course, there is no formal SRE certificate that is widely recognized so who qualifies to be an SRE tends to vary from one organization to the next. The one thing that is generally agreed upon is that SRE automate a wide range of tasks that would otherwise require multiple IT administrators to perform. Glassdoor also reports that, on average, an SRE makes north of about $125,000, which ranks among the highest salary level within any IT organization.

Each organization, depending on their geographic region, would need to determine how many IT administrators they could replace if they hired an SRE. That doesn’t mean IT administrators that rely on a wide range of graphical tools to manage IT tasks are going away any time soon, but as DevOps continues to evolve, the dynamics between SRE and IT administrators also will continue to evolve.

November 8, 2022
Snowflake Makes Case for Building and Deploying Apps on Platform

During a Snowday 2022 event, Snowflake this week moved to make it simpler to build applications on top of its cloud platform for managing data in addition to enhancing the overall security of the platform.

The company has made generally available Snowpark, a framework for building Java, Scala, SQL or Python applications. Snowflake also committed to making generally available in early 2023 an open source Streamlit framework for building Python applications that Snowflake acquired earlier this year.

Snowflake also revealed it is adding observability tools to the platform to make it easier to build, test, debug, deploy and monitor data pipelines, including alerting tools (private preview), logging capabilities (private preview), event tracing (private preview), task graphs and history (public preview).

Torsten Grabs, director of product management for Snowflake, said those capabilities will make it easier to alert DevOps teams to issues that might arise with applications deployed on the Snowflake platform.

At the same time, Snowflake is also making it simpler to build data pipelines by making a Serverless Tasks capability generally available. In addition, the company is previewing a Schema Inference engine that makes it simpler to ingest data and a Dynamic Tables capability to declaratively manage data pipelines.

Other capabilities being added to the platform include a public preview of a Query Acceleration Service to make additional compute resources available on demand. Snowflake has also improved query efficiency with join elimination in addition to providing search optimizations.

There are also now cost optimization tools that enable IT teams to run a cost-benefit analysis of Snowflake services in addition to tools that make it easier to replicate data across multiple cloud computing environments running the Snowflake platform.

IT teams can now also replicate Streams and Tasks via a capability in public preview that enables data pipelines to seamlessly work on secondary Snowflake accounts and configure failovers using a graphical tool that is available in private beta.

Finally, Snowflake has also enhanced its cross-cloud Snowgrid Capability to make it easier to share both public and private listings of data in addition to applying tags to and searching masked data.

The latest editions to the Snowflake platform come at a time when more organizations are starting to adopt DataOps best practices to manage data. Snowflake is now making a case for building and deploying applications on top of a platform that automates the management of the data pipelines used by applications to access data.

It’s not clear to what degree the Snowflake platform will be employed to host applications, but the company is clearly betting the volume of data residing in its cloud platform will attract a wide range of application developers that, in turn, will need to be supported by DevOps teams.

Of course, Snowflake is not the only provider of a data warehouse or data lake that has similar ambitions. Regardless of platform, however, the one thing that is clear is that a convergence of DevOps and DataOps workflows is now underway as IT organizations look to streamline the management of the massive amounts of data that now reside in the cloud.

November 8, 2022
Cisco Adds Open Source Tool to Validate Serverless Functions

Cisco has launched an open source project, dubbed FunctionClarity, that makes it possible to verify signatures before code is deployed in a serverless computing environment.

Vijoy Pandey, vice president of emerging technologies and incubation at Cisco, said that one of the application security issues that has emerged in serverless computing environments is the lack of validation capability to verify code signatures prior to deployment. As more organizations employ serverless computing frameworks, the overall software supply chain becomes less secure, he noted.

FunctionClarity includes a command line interface (CLI) tool to invoke a verification function that can be deployed on the Amazon Lambda serverless platform to address that issue; support for Microsoft Azure and Google Cloud serverless platforms is planned, he added.

The FunctionClarity project aims to close that gap by employing either a keyless approach using open source sigstore tools or a traditional encryption key, noted Pandey.

The FunctionClarity project is the latest extension of a Cisco OpenClarity initiative that includes an existing project for securing application programming interfaces (APIs) using a service mesh, dubbed APIClarity, and a tool for detecting and managing software bills of materials (SBOMs) and vulnerabilities in container images and filesystems dubbed KubeClarity. All three projects are part of an ongoing Cisco effort to contribute code to the open source community that, while essential, does not provide a lot of differentiated value for any one vendor, said Pandey.

The FunctionClarity project arrives at a time when concerns over the security of software supply chains is on the rise. Many of the software security issues that organizations are encountering stem from the simple fact that developers can provision cloud infrastructure without any kind of meaningful cybersecurity review. Code signing, within the context of a larger set of DevSecOps workflows, is now starting to gain traction to better ensure secure software supply chains. The challenge is that as more code winds up running on serverless computing platforms, a need to extend the reach of whatever approach to code signing an organization adopts becomes more evident.

In general, the open source community is collectively trying to respond to an executive order issued by the Biden administration that required federal agencies to implement a series of steps to better secure their software supply chains. One of those measures includes wider adoption of code signatures to ensure the authenticity of code being incorporated into applications. The challenge is that incorporating code signatures into DevOps workflows has historically not been a trivial process to implement. Most recently, a free sigstore cloud service launched that promised to make it easier to achieve that goal.

It’s not clear how widely or even how rapidly code signing will be adopted, but enterprise IT organizations are paying close attention to how federal agencies are responding to the executive order. Many of the tools and processes defined by those agencies will inevitably be copied by enterprise IT organizations that have thus far made limited use of code signing to verify the authenticity of software components. The challenge now is finding a way to achieve that goal without slowing down the rate at which modern software is developed.

November 8, 2022
Developers Continue to Prefer Remote Work

A survey found 83% of developers are working remotely more than half of the time, with more than three-quarters (76%) reporting they have maintained or increased the frequency of remote work in the last year. In fact, three in five respondents (62%) work remotely at least 90% of the time.

The survey, published by Netlify, a provider of a platform for building the front end of Web applications using the Jamstack framework, polled 6,544 developers in the Jamstack community and found more than half (55%) would quit their jobs if forced to return to an office.

It’s not clear how much remote work has become a de facto requirement for organizations to attract and retain developer talent, but it’s clear that, in the wake of the COVID-19 pandemic, the dynamic between developers and the organizations that employ them has completely changed.

Netlify CEO Mathias Biilmann said regardless of industry, it’s apparent that DevOps teams are going to continue to need to make adjustments to foster collaboration across distributed teams. Senior managers will find it increasingly challenging to manage workflows across a workforce that is also witnessing a lot of turnover, he noted. A third of survey respondents (33%) reported having changed jobs in the last year. It’s not clear how much a lack of interaction in the office might be impacting innovation, but at a time when demand for developer talent far exceeds the available supply, organizations will need to adjust to what appears to have become a permanent new reality.

At the same time, the Netlify survey noted the pace at which emerging technologies are being embraced continues to rapidly evolve—perhaps because of rather than in spite of remote work. For example, the survey found 70% of developers reported using a serverless computing framework in the last year. Nearly half (47%) are also experimenting with edge dynamic sites, the survey found. It’s clear that company culture, at least in terms of technology adoption, plays a large role in attracting developer talent.

Overall, 96% of survey respondents noted that JavaScript is their primary choice of programming language, with 21% of developers now specifically using TypeScript to build applications. The React framework for building applications, meanwhile, is being used by 71% of developers, with 47% reporting they employed the Next.js version of that framework. Other React frameworks gaining traction include Vite (32%), Sveltekit (15%), Astro (11%), Remix (10%) and SolidJS (6%).

However, emerging Web3 technologies have yet to gain traction. While 7% to 10% of developers have tried out technologies such as cryptocurrencies and NFTs, only 3% reported using these technologies regularly. A total of 42% of developers either didn’t know what Web3 was or didn’t care about it, while 31% felt negatively about it, the survey found.

Ultimately, the primary reason organizations invest in DevOps is to increase developer productivity. The challenge is that, as developers embrace more tools and platforms for building applications, the process of deploying those applications becomes more challenging to manage effectively. One way to achieve that goal is to separate frontend and backend software development processes to enable developers to focus more on writing code rather than managing the arcane processes required to deploy it.

November 7, 2022
2023 Application Security Budgets on the Rise

A survey of 500 DevSecOps professionals in the U.S. found nearly three-quarters (73%) of organizations plan to increase investment in application security in 2023.

The survey, conducted by Wakefield Research on behalf of Invicti, a provider of dynamic application security testing (DAST) tools, also found 97% of DevSecOps teams said they ignored a real vulnerability at least once a month because they assumed it was a false positive.

Invicti CTO and head of security research Frank Catucci said that while it is apparent organizations are under pressure to improve the security of their software supply chains, the tools and processes used to discover vulnerabilities need to be revisited at a time when the security technical debt accrued in production environments is already massive.

Many DevOps teams have routinely shipped code with known vulnerabilities for years, noted Catucci. They are not only under pressure to improve the security of new applications but also remediate applications with known vulnerabilities already running in production environments, he added.

Cybersecurity teams have, of course, been identifying those vulnerabilities for years. The challenge is that the long list of vulnerabilities cybersecurity teams provide to DevSecOps teams tends to lack any context. As such, it becomes impossible for developers to prioritize their remediation efforts based on the actual severity of the vulnerabilities found. DAST tools help address that issue because they provide insights into vulnerabilities in applications already running in a production environment.

Application security, of course, has historically been problematic because many developers assume cybersecurity teams are securing the environments where applications are deployed. Most cybersecurity teams, however, allocate their budgets to tools and platforms they control and manage. Most of those tools and platforms are focused on security operations tasks that tend to have more to do with securing infrastructure rather than applications. The assumption is application development teams are responsible for securing the software that runs on those platforms. That disconnect is now finally being addressed as responsibility for application security shifts left to become an integral part of DevOps workflows.

The Invicti survey suggested that application security budget allocations are now catching up with that transition. In fact, the Invicti survey found that 100% of DevSecOps professionals are tracking the return on investment (ROI) on the application security tools and platforms they employ. Of course, measuring the effectiveness of any cybersecurity tool or platform has always been a challenge. It’s never quite clear whether a breach didn’t occur because a tool or platform was present or if it didn’t happen because of sheer luck and happenstance.

Regardless of the ROI achieved, however, it’s clear that cybercriminals are increasingly targeting software supply chains. The recent executive order issued by the Biden administration requiring federal agencies to review the security of their software supply chains got the attention of traditional enterprise IT organizations, as well. As such, the overall state of application security should steadily improve in the months and years ahead, as more funding is allocated to prevent vulnerabilities from finding their way into production environments in the first place.

November 7, 2022
Five Great DevOps Job Opportunities

DevOps.com is now providing a weekly DevOps jobs report through which opportunities for DevOps professionals will be highlighted as part of an effort to better serve our audience.

Our goal in these challenging economic times is to make it easier for DevOps professionals to advance their careers. Of course, the pool of available DevOps talent is still relatively constrained, so when one DevOps professional takes on a new role it tends to create an opportunity for others.

The five job postings shared this week are selected based on the company looking to hire, the vertical industry segment and naturally, the pay scale being offered.

We’re also committed to providing additional insights into the state of the DevOps job market. In the meantime, for your consideration:

ZipRecruiter.com

Comcast
New York City
Lead Software Engineer – DevOps
$190,950 to $225,278

Indeed.com

Divi Technologies
Remote
DevOps Engineer
$140,000 to $155,000

SimplyHired

Perimeter 81
New York City
DevOps Engineer
$130,000 to $175,000

Dice.com

OnDemand Group
Minnetonka, MN.
Senior DevOps Engineer
$120,000+

LinkedIn

Gainwell Technologies
Remote
Senior DevOps Engineer
$92,500 to $115,600

November 7, 2022
OpenSSL Fiasco: What can DevOps Learn? | Elon Fires ‘50%’ of Twitter

Welcome to The Long View—where we peruse the news of the week and strip it to the essentials. Let’s work out what really matters.

This week: The OpenSSL project has egg on its face, and half of Twitter’s staff are for the chop tomorrow. (more…)

November 3, 2022