devopsy.kinsta.cloud

Category: DevOps Toolbox

Eclipse Foundation Opens Marketplace for OpenJDK Binaries

The Eclipse Foundation this week opened an Adoptium Marketplace through which DevOps teams can access Java binaries based on the OpenJDK specification. OpenJDK was created based on the Java standard edition (SE) of the programming language and virtual machine.

Mike Milinkovich, executive director of the Eclipse Foundation, said the goal is to make it easier for IT teams to access open source binaries via a vendor-neutral marketplace that complies with the Java SE Technology Compatibility Kit (TCK) that was created to make sure runtimes based on OpenJDK conform to the original Java specification.

The marketplace has been set up in collaboration with the Adoptium Working Group that was set up by the Eclipse Foundation to govern OpenJDK, an open source implementation of the Java SE originally created at Sun Microsystems and later acquired by Oracle.

Members of The Adoptium Working Group include Alibaba Cloud, Azul, Huawei, IBM, iJUG, Karakun AG, Microsoft, New Relic and Red Hat. That group created an Eclipse AQAvit project to test binaries for compliance with the Java SE TCK specification.

OpenJDK is a reference implementation used by multiple vendors to create their own binaries, which are only guaranteed to be compatible with the standard until the next update arrives; updates are released approximately every six months. The Eclipse Adoptium project will make it simpler for developer teams to identify which binaries are compatible with specific implementations of OpenJDK. Ultimately, the goal is to encourage developers to build applications using frameworks that make use of new OpenJDK features as they are added to the platform versus continuing to rely on older binaries.

Java remains the dominant language for developers working in enterprise IT organizations. Over the last few years, however, OpenJDK has gained traction as organizations adopted an open source implementation of the specification that enabled providers of application development frameworks to reduce licensing costs.

It’s not clear at what rate enterprise IT organizations are replacing proprietary editions of Java with OpenJDK. However, during an economic downturn, more organizations tend to rely on open source software. Many organizations are also now looking to build microservices-based applications using Java frameworks based on OpenJDK runtimes.

It’s obviously taken quite a bit of time for the OpenJDK community to put together a vendor-neutral marketplace. In the meantime, a wide range of other programming languages gained traction in the enterprise. There is still a lot of dependency on Java for building backend applications and services, but it’s more common than ever to see frontends that access those backend applications and services being built using, for example, a framework based on JavaScript.

Java, of course, isn’t going away any time soon. The sheer number of enterprise developers that rely on it to build applications will ensure its continued longevity. The important thing to note, however, is that as more open source platforms are employed, the pace at which the overall community will innovate should continue to accelerate.

May 27, 2022
Opportunities and Challenges of Observability

Observability platforms that unify metrics, logs and traces across applications and IT infrastructure are transforming how IT is managed. Most DevOps teams today are able to continuously monitor IT environments using tools that track a set of pre-defined metrics; observability platforms make it possible to aggregate data so that DevOps teams can launch queries that help them uncover the root cause of IT issues. DevOps teams leverage observability to allow them to understand what is happening within their applications more effectively and efficiently.

Historically, applications were black boxes—meaning it was (and still is, in many cases) extremely difficult to know what is happening under the hood of the application.

With the advent and subsequent growth of observability tools and techniques in the DevOps space, teams went from struggling to collect and understand the available data (which could vary from robust to almost non-existent) to easily aggregating data from numerous sources and being able to understand what that data means to being completely deluged with data—so much data it’s hard to keep up.

Gather Data is No Longer Enough

Josh Chessman, vice president of strategy and innovation at IT infrastructure monitoring specialist Netreo, explained that as applications become more and more complex, (e.g., running across multiple disparate environments with different technologies and in different locations) being able to collect data from these different environments is no longer enough.

“Instead, DevOps engineers need to be able to collect the data and efficiently analyze it, too, to understand what is happening within the application,” he said. “Observability is the concept that drives this.”

Chessman pointed out there are numerous observability issues DevOps teams face, but that they vary from organization to organization (and sometimes from team to team within an organization).

“Part of the problem is that many organizations simply lack tools to provide the data collection and analysis needed to move to an observability paradigm,” he said.

This could be for numerous reasons, including budget (if you already have an existing tool, can you afford to acquire a new tool?), functionality (not all tools that claim to provide observability really do), implementation (this stuff is not easy and a poorly implemented tool could actually make things worse) and more.

“One of the larger challenges I’ve seen is that organizations will go out and acquire an ‘observability’ tool and assume that is all they have to do,” Chessman said. “Unfortunately, that is not the case. Acquiring the tool gets you the technology, but the team must also be structured and ready to take advantage of the concepts and functionality provided by the tool.”

GitLab’s senior director of product management Kenny Johnston explained that, on average, DevOps teams use five different observability tools which tend to inadequately connect developers to observability.

In turn, developers struggle to respond to incidents swiftly while jumping across multiple tools and sources of truth and struggle to assign service level objectives (SLOs) and error budget definitions to the responsible product development teams.

“To remedy this, we recommend adopting a unified software development solution that replaces the DIY ‘duct tape’ of products constraining developers and helps them deliver software more securely, at speed and scale,” he said.

Johnston said the most flexible and future-proof solutions today are open source tools like Prometheus and its ecosystem, as well as the OpenTelemetry toolset for more advanced use cases such as tracing.

Prometheus has become very common in the developer space and, unlike other open source solutions or libraries, its format automatically emits metrics.

“This allows for simple and future-proof interoperability, as one of the hardest things to change later down the line is code running,” he said. “Using open standards reduces the need for developers to rewrite and reintegrate the same type of code over and over again.”

Observability Requires Understanding Your Goals

From Chessman’s perspective, understanding what your goals are is critical for observability.

“Observability tools can do many things, but their primary function is to collect and analyze data,” he said. “While that is all well and good, if you do not define your KPIs from the beginning, all the collected and analyzed data is meaningless since you have no concept of what good and bad are.”

Trying to define good and bad from the data collected is likely to result in failure because you are looking at where you are to figure out where you want to go.

KPIs vary from application to application and, if you do not have them identified and well-defined before beginning down the observability path, it is that much harder, he said.

Most tools (observability or otherwise) are capable of collecting hundreds if not thousands of metrics, but not all—not even most, in many circumstances—will be relevant to an organization.

“Instead of having to sift through the cruft to find the valuable data, it is better to start with identifying the valuable data and then figure out how to get there,” Chessman said. “If you don’t identify your KPIs in advance you could end up using the wrong KPIs for the wrong scenarios.”

Johnston added that KPIs, like DevOps, are an iterative process and starting with KPIs helps developers create systems that can meet performance and availability expectations.

“It’s important that DevOps teams continuously review their KPIs to allow for regular feedback on successes and failures,” he said.

From his perspective, the best place to start for maintaining and improving observability is by storing observability definitions, instrumentation, SLOs, thresholds, incident response templates and dashboards as code.

“This allows for rapid improvement via code changes by any team member, whether that’s a site reliability engineer (SRE), developer or platform team,” he said.

May 27, 2022
Go Language is Popular? | VMware Sells to Broadcom? | Unlimited PTO?

Welcome to The Long View—where we peruse the news of the week and strip it to the essentials. Let’s work out what really matters.

This week: Golang goes from zero to hero, will Broadcom buy VMware? And limitless vacation is still a Thing. (more…)

May 25, 2022
JFrog Launches Blockchain Project to Secure Open Source Software

At its swampUP event, JFrog today launched Project Pyrsia, an open source project that uses a blockchain platform and Sigstore Cosign and Notary V2 cryptographic signature software to secure software packages. In addition to JFrog, other contributors to the project include Docker, Inc., DeployHub, Futureway and Oracle.

Stephen Chin, vice president of developer relations for JFrog, said Project Pyrsia will enable organizations to establish a chain of provenance for open source software components stored in a secure network of repositories.

In effect, Project Pyrsia is making use of decentralized Web3 technologies to secure the open source supply chain, noted Chin. That approach to validating the integrity of software components using a blockchain platform will ensure that any software component being employed by developers has not been compromised, he added.

Ultimately, the goal is to contribute Project Pyrsia to the Open Source Security Foundation (OpenSSF), an arm of the Linux Foundation that, as a consortium, is looking to coordinate efforts to better secure open source software. JFrog’s own research efforts identified more than 20 different open source software supply chain attacks, with two of those involving zero-day threats for which there was no immediate software patch available. Cybercriminals are targeting open source projects because any malware that gets included will later show up in any number of downstream applications. Their ultimate goal is to activate that malware at a time of their choosing.

Securing open source software became a more urgent issue following the discovery last year of the zero-day Log4Shell vulnerability that impacted Java applications. Many developers routinely reuse open source software, but many of those projects are maintained by a small number of programmers that voluntarily contribute their time and effort to build components that others are free to use. Like any other developer, the amount of security expertise those individuals have is limited; the onus for making sure that software is secure falls on the organizations that decide to deploy it. The trouble is, many developers assume that software is more secure than it really is. Initiatives like Project Pyrsia are part of a larger effort to make it simpler for maintainers to secure open source software.

It’s not clear whether security concerns are prompting organizations to review the amount of open source software they consume. Most organizations are more dependent on open source software than they realize, because most packaged applications will include open source components. Whenever a zero-day vulnerability is discovered, organizations can spend months looking for all the instances of an open source component that might be vulnerable.

In theory, increased focus on open source software should lead to greater adoption of DevSecOps best practices that reduce the number of vulnerabilities in production environments. In the meantime, more scrutiny of open source software components is necessary, considering that they are employed by almost every organization.

May 25, 2022
WhiteSource Becomes Mend, Launches Automated Remediation Platform

WhiteSource rechristened itself Mend today and launched a remediation platform that automatically resolves security issues for application developers.

Rami Sass, co-founder and CEO of Mend, said now the company is going beyond just identifying vulnerabilities in open source software using software composition analysis (SCA) tools and is also fixing them. The overall goal is to make it simpler for developers to address security issues without taking time away from writing code or slowing down the rate at which applications are developed, he added.

To further that effort, Mend also announced today it is making available a plug-in for JFrog Artifactory that enables Mend Supply Chain Defender, formerly WhiteSource Diffend, to enable detection of malicious open source code. Last year the company acquired Diffend followed by the acquisition of two startup providers of static analysis security testing (SAST) tools.

Collectively, those applications enabled the company to build the Mend Application Security Platform, a software-as-a-service (SaaS) offering that combines SCA and SAST tools to create an automated remediation framework that can be applied to both open source and proprietary code residing in a repository.

Mend claimed it added more than 350 customers in the last year to bring its total to more than 1,000 organizations. Most recently, the company raised an additional $75 million in financing to continue investing in a platform specifically designed to address application security issues. In the wake of a series of high-profile security breaches involving software supply chains, Sass noted there is now a much greater appreciation for securing applications and that demand for application security solutions is growing. That demand, in turn, is fueling a wave of consolidation that is, in part, enabled by the convergence of application security tools made available via a SaaS platform, he added.

Historically, much of the focus on application security focused on discovering vulnerabilities that developers are asked to patch. The issue is that developers are being asked to patch the same modules repeatedly. The Mend Application Security Platform keeps track of what modules have been successfully updated to give developers higher confidence in the updates being applied, said Sass.

Those recommendations are not being surfaced using machine learning algorithms but rather by the data analytics capabilities that have been added to the company’s portfolio of tools over time, he noted.

While there may never be such a thing as perfect security it’s apparent that most application security issues can be traced back to relatively common mistakes that developers routinely make. The more those issues are surfaced within a developer workflow, the less dependent organizations will need to be on embedding guardrails within DevSecOps workflows to prevent vulnerabilities from finding their way into production environment. The Mend Application Security Platform doesn’t eliminate the need for those guardrails as much as it reduces the sheer volume of security issues that might otherwise need to be addressed.

It may be a while before these more advanced developer security tools achieve that result, but it’s clear from how quickly advances are being made that such tools should have a material impact on the overall state of application security.

May 25, 2022
GitLab Gets an Overhaul

GitLab launched its next major iteration, GitLab 15, starting with its first release version, 15.0, which the company said pulls together new DevOps and data science capabilities into the platform. With GitLab 15, GitLab says it provides (or soon will provide) continuous security and compliance, enterprise Agile planning, visibility and observability, workflow automation and increased support for data science workloads.

In an interview with devopsy.kinsta.cloud, Hillary Benson, director of product management, detailed how GitLab 15 provides several security benefits, including the ability to automatically create an exportable software bill of materials (SBOM) and signed attestation for build artifacts, centralized security policy management including a two-step exception approval process, as well as providing compliance managers the ability to see project history changes and better enforce policy. “Supply chain security, especially, has become a major area of focus for us, both because it’s become a high priority in the industry and because GitLab is uniquely positioned to support customers well in this area,” Benson said.

GitLab 15 also provides improved static application security testing (SAST) with stronger and more flexible rules and finding management, Benson said. Finally, users can pool together GitLab project data to create custom automations.

According to Benson, GitLab 15 will also introduce more capabilities that interlink data across every stage of the DevOps life cycle from initial planning, implementation, deployment and monitoring. “Some of the things that will happen in this area are additional options for the types of work items that can be managed,” said Benson. That includes supporting a more diverse range of workflows and frameworks—including enterprise Agile frameworks. The platform will continue to support issues, epics and requirements and soon enable tasks, features and objectives.

GitLab 15 provides new capabilities the company hopes will increase visibility and enable an end-to-end view of value delivery and application health, the company said. The unified collaborative context will help remove DevOps silos within organizations and decrease incident rates as well as provide comprehensive insights into issues that hinder performance in addition to real-time triaging of incidents. These new capabilities can help shorten lead times from code to production, reduce error frequency and severity, help development teams deploy more frequently and reduce mean-time-to-recover (MTTR) after an incident.

Upcoming features include improved product analytics and experimentation, such as increasing the practicality of surfaced data by enabling testing and validating new ideas and measuring the adoption of DevOps best practices across teams and projects, Benson said. Expanded support for all DevOps research and assessment (DORA) metrics including improvements to value stream analytics for projects and groups is also addressed. “This is about making it easier to understand what value your product is bringing,” said Benson. “We’re introducing product analytics and experimentation to increase the practicality of the data that you’re using and the data you’re collecting to actually move your product forward,” says Benson.

Finally, GitLab 15 will increase support for data science teams and their workloads. “This is about streamlining the development and deployment of machine learning models to train and test those and then get them into production. We’ll also be working on providing monitoring for those workloads so that it can be understood if those models are being used, how they’re being used and where there may be performance issues,” said Benson.

May 23, 2022
DevSecOps Deluge: Choosing the Right Tools

In the last few years, DevSecOps has become the security process of choice for many forward-thinking enterprises.

These organizations have come to understand that fixing bugs in the latter stages of product and application development offers no favors to anyone but cybercriminals. So, they have overhauled traditional processes and united development, operations and security teams to form DevSecOps where the teams now work in tandem, cooperating to embed security into the entire process of application and software development.

The benefits of this are clear; finding and fixing bugs early closes doors on cybercriminals while bringing peace of mind, cost savings and better products to customers. However, one of the challenges many organizations face when transitioning to DevSecOps is making the process seamless, ensuring it doesn’t slow down the software development life cycle, make security checks more cumbersome or cause frustration among teams.

Security teams need to ensure their DevSecOps program is delivering value to the culture of the organization while speeding up time-to-fix, capturing the right metrics, and categorizing and prioritizing issues based on their risk so that nothing goes live that could cause major damage.

Additionally, they also need to select the correct DevSecOps toolkit to carry out application security testing while ensuring the tools easily integrate into software development and can be used across multiple projects. When identifying tools for testing, DevSecOps teams are often overwhelmed by the huge range and volume available, which makes choosing which ones to use and learning how to use them a minefield, even for those with the right training and know-how.

A DevSecOps automation program needs lots of technical tools and cultural aspects to come together to make it work. Security teams need static analysis tools to check the code, third-party library analysis to check the dependencies, separate analysis to check the infrastructure-as-code (IaC) configuration, scanners to check containers for issues, tools to test the running system, cloud security checks and infrastructure testing for patches and ports. They also need to match these tools up to the right technology each team is using and keep up with the constant changes.

Given all of these complexities, how can DevSecOps teams overcome these challenges and build out an effective DevSecOps program with the correct toolset?

Here are some top tips to help them along the way.

Keep Security Processes Flexible

Your teams are going to be using different technology stacks, different languages, frameworks, and so on. If you tie your process too tightly to a few tools, it’ll be harder to slot in new checks when things change. Remember that the aim is a consistent, repeatable security process with the right visibility—the technical tools feed into that, but they are not the whole process. What security checks will you need in 12 months that you don’t have now? What about 36 months? These are questions that must be asked continuously.

Automation is Your Friend

If development pipelines are running smoothly and automatically, then any manual security steps are not going to fit in with the process. Automating security tools together—from orchestrating their running to aggregating their responses and managing issues—will save a lot of time and give you the results you need in line with your releases.

Keep an Eye On Your ROI

It’s very common for big-ticket commercial tools to be underused; you have the licenses for repeated testing and you have frequent code drops, but the processes or the integrations mean testing isn’t applied as often as needed or the results take time to collect and process. Explore ways to bring commercial tools easily into the existing processes.

There’s No Such Thing as a Free Lunch

The open source community provides great security tools, but remember: There is a cost associated with the time it takes your teams to use them and to manage the outputs. From developers learning how to run them to the time it takes to actually run them or extract the results or manage false positives—these aren’t free. A ‘free’ tool that costs two hours of work every release might not be worth it.

DevSecOps is All About Collaboration

Weaving security checks into development processes, CI/CD pipelines, ticketing systems and sprint meetings help security to be part of the development process, but not every developer is—or wants to be—a security expert. Think about how the aims of development and security can meet in the middle. How can developers include security without much effort or thought? How can security help supply the tooling and help triage issues in developer pipelines? How can the process conform to auditors’ tools and with correct profiles, not just turned down to zero to avoid reporting issues? How can security add value at scale to help developers fix issues faster regardless of the tooling being used?

May 20, 2022
New Relic Expands Scope of Observability Reach

At its Futurestack conference, New Relic announced it expanded the integrations and tools it provides for its observability platform and added its first cybersecurity tool.

The company now provides more than 470 integrations with cloud services, open source tools and other enterprise technologies, with support for offerings from Akamai, Atlassian, CircleCI, Cloudflare, Netlify, PagerDuty and Postman now being added to its Instant Observability platform. The company has also added the ability to now collect logs alongside metrics and traces within its Application Performance Monitoring (APM) offering.

Finally, New Relic has deepened its cloud ties with Microsoft, added a guided user onboarding interface, support for a wider range of instrumentation methods, tighter integration with Kubernetes and, for the first time, is branching out into the realm of cybersecurity via a New Relic Vulnerability Management service that is being added to its software-as-a-service (SaaS) portfolio.

Ishan Mukherjee, group vice president for product go-to-market at New Relic, said New Relic Vulnerability Management is the start of a larger effort to enable IT teams to leverage the observability data they already collect via the New Relic platform to drive the adoption of DevSecOps best practices. Legacy approaches to vulnerability management create yet another silo of data that only further aggravates cybersecurity blind spots, he noted. New Relic Vulnerability Management aggregates data collected via its agent software and integrations with a wide range of platforms to provide a unified view of the overall security posture of an IT environment, Mukherjee added.

It’s still early days as far as adoption of observability platforms is concerned, but it’s apparent that platforms that unify the collection of metrics, logs and traces across both applications and the IT infrastructure they run on are transforming how IT is managed. Most DevOps teams today are able to continuously monitor IT environments using tools that track a set of pre-defined metrics; observability platforms make it possible to aggregate data so that DevOps teams can launch queries that help them uncover the root cause of an IT issue. That capability is increasingly becoming critical as IT environments—thanks, in part, to the rise of microservices and cloud computing environments—become too complex for IT teams to manage using legacy tools.

The challenge is that most IT teams are not quite sure what questions they should ask to get the most value from investment in an observability platform. Observability platforms that accurately surface issues without requiring much intervention from IT teams are likely to gain the most traction, as the competition among providers of these platforms intensifies.

In the meantime, organizations might want to start reevaluating roles within their IT teams as it becomes simpler to collaborate around a single source of data. The days when IT teams had to triage issues using disparate tools that surfaced conflicting data may finally be coming to an end. As such, it should become possible for a team of IT professionals to manage a wider range of processes across an extended enterprise IT environment.

Naturally, it may be a while before observability platforms drive wholesale reorganizations of IT teams, but it’s now more a question of when rather than if that day will come.

May 18, 2022
Is Your Future in SaaS? Yes, Except …

The growth of SaaS—including IaaS—was phenomenal when it came and it has become somewhat pervasive. Today, there are SaaS tools in use at just about every business, large or small. From sales to marketing to customer management, there is a SaaS tool that does it affordably and well. For now. The same is true of cloud vendors. Cloud prices are kept low by two mechanisms—volume and profit. Volume; much like the theory that mass transit is cheaper, by putting many customers on a single server reduces cloud vendors’ operating expenses and thus reduces what the cloud vendor must charge to make a profit.

But as costs rise and growth slows in the current environment, it is important that your IT department, as a customer, pays attention to the cost/benefit equation of all SaaS offerings. Some will remain beneficial because of the work they offload, even as prices increase for your organization. Others, you’ll see, will quickly outgrow the value they bring as costs rise and there is a third class that just isn’t worth it—maybe it never was and you haven’t bothered to pay attention.

Look for those items. So that this is less vague, here’s an example: A company I am involved with has been paying for a SaaS service that simply emulates an on-site functionality. The organization could (and previously did) drop a couple of boxes in and achieve largely the same result. Recently, we looked more closely at items like this and found that, in this particular case, (though not in most SaaS cases the org is using) the cost of dropping two boxes in and letting them do the job was cheaper (assuming those boxes were upgraded/replaced every other year) than continuing with the SaaS solution at the current monthly price. So much cheaper, in fact, that a contractor could be hired to deal with any issues that came up with the physical boxes and the only increased work for IT would be maintenance updates to keep the system secure. If, instead of appliances, the solution consisted of two Windows or Linux boxes, even that maintenance cost would be minimized.

Service will be the same or better—the devices are right there if a change is needed—and there will be one less link in the chain of tools/vendors for attackers (the physical solutions would be on a private subnet, SaaS cannot be). Assuming the solution is a quality product, there really is no downside here. And yet, for several years this company has used the SaaS solution. These are the low-hanging fruit that people will point at and call “retrenchment” or whatever. Ignore them. Do what’s best for your org—be it SaaS or local—and make that decision on a per-tool/per-project basis.

And keep kicking it. You’re building apps and infrastructure that keep the company running. Don’t hesitate to take pride in what your team(s) have built and don’t hesitate to cut costs on things that don’t matter so you have more funds for things that do. And once again, thank you—from all the employees in your organization that don’t realize they should say thanks.

May 18, 2022
Apple Allows 50% Fee Rise | @ElonMusk Fans: 70% Fake | Microsoft Salaries up by 100%?

Welcome to The Long View—where we peruse the news of the week and strip it to the essentials. Let’s work out what really matters.

This week: Apple lets devs increase subscriptions “without consent,” Twitter has more than 19% spam accounts, and we dig into the claim that Microsoft is about to double salaries. (more…)

May 17, 2022