devopsy.kinsta.cloud

Tag: open source

Survey Sees Surge in Demand for DevOps Skills

The Linux Foundation and edX, a provider of an online learning platform, jointly released an annual report today that for the first time finds organizations are looking for more IT personnel with open source DevOps skills (65%) than developers (59%).

Clyde Seepersad, senior vice president and general manager for training and certification at The Linux Foundation, said the 2020 Open Source Jobs Report shows that as more workloads shift to the cloud in the wake of the COVID-19 pandemic, the number of organizations looking to embrace best DevOps practices is increasing substantially. The report finds 70% of respondents are more likely to hire an IT professional with cloud computing skills.

Based on a survey of 175 hiring managers and 900 IT professionals with open source expertise, the report also finds that despite the economic downturn brought on by the pandemic, 37% of hiring managers say they will be looking to hire more skilled IT professionals in the next six months.

The report also finds 93% of survey respondents report difficulty finding open source talent. An overwhelming 81% of hiring managers said recruiting open source talent is a priority for 2020, with 56% looking to recruit IT professionals with open source skills in the next six months. Nearly two-thirds (63%) said their organizations have begun to support open source projects with code or other resources for the explicit purpose of recruiting individuals with open source software skills.

More than half of hiring managers (52%) said they are more likely to hire someone with a certification, while 80% of survey respondents reported providing online training courses for employees to learn open source software. Nearly three-quarters of organizations (74%) are now offering to pay for employee certifications, according to the survey results.

An equal percentage of hiring managers also identified Linux as the most in-demand skill they seek in new hires, while 63% also said they are looking for IT architects who can build solutions based on open source software.

Seepersad said given the difficulty attracting talent many organizations are focusing more on training their existing staff rather than trying to recruit talent in an open market where demand for open source expertise remains high.

The talent pool is more global as more organizations become accustomed to working with staff who are now largely working from home, which, coupled with current economic realities, is limiting how much organizations are willing to pay for IT talent. More organizations are also keen to make sure there is a cultural fit with their organization when hiring, he added.

In general, it’s clear more IT organizations will need to focus on growing their own farm teams with open source DevOps skills rather than hoping to lure away superstars from other organizations. Rather than simply trying to fill open positions, organizations need to think more in terms of acquiring and managing talent in the same way a general manager manages a sports franchise.

October 26, 2020
Azul Adds OpenJDK Migration Services

Azul today announced today it will provide migration services for IT organizations looking to transition from the proprietary Oracle Java Standard Edition (SE) developer platform to the Zulu edition of an open source OpenJDK equivalent.

OpenJDK is based on the HotpSpot edition of the Java virtual machine, the Java Class Library and the Java compiler. The Zulu edition was originally developed by Azul and Microsoft and has since been supported by Oracle, IBM, Apple, Amazon, SAP and Red Hat.

Azul CEO Scott Sellars said that while OpenJDK is compatible with Oracle Java SE there are some minor differences that enterprise IT organizations need help with to make the transition. While distributions of OpenJDK have been around for years now, Sellars said the fees for support that Oracle has been charging for Oracle Java SE have reached a point where more organizations are now ready to make the jump to OpenJDK.

In the wake of the economic downturn brought on by the COVID-19 pandemic organizations of all sizes are looking to reduce costs by transitioning to open source software whenever possible, he added. Rather than reduce the size of their IT staff most organizations would prefer to rein in their IT budgets by reducing their dependency on proprietary software, Sellars noted. The United States Transportation Command (USTRANSCOM), for example, earlier this month revealed it is replacing Oracle Java SE with Azul Zulu builds of OpenJDK to in part lower Java support costs. Taboola, an online advertising platform, also made a similar switch.

Azul is now offering two classes of migration services. A Level 1 offering helps organizations migrate directly from Oracle Java to Azul Zulu builds of OpenJDK and includes creating an inventory of the Java estate by vendor, by Java version, by Java security patch level and by which Java runtimes are currently being employed. A Level 2 service brings in a third-party IT services firm to enable organizations to migrate older implementation of Java Virtual Machines (JVMs) to either the Java 8 or Java 11 platform.

A recent Forrester Consulting study commissioned by Azul finds the Azul Zing Java Virtual Machine pays for itself in less than three months by reducing development and infrastructure costs, with a three-year 224% return on investment (ROI).

Regardless of the open source instance of a Java platform, it’s clear that going forward most of the ongoing innovation is shifting to platforms that multiple vendors are now contributing to more frequently. OpenJDK, for example, is now on a release cadence of every six months. Each distribution of an open source Java platform serves to reduce costs in a way that enables IT organizations to achieve backward compatibility.

Despite first being employed in the enterprise more than two decades ago, reliance on Java platforms remains high. There has been widespread adoption of multiple programming languages but most back-end systems in the enterprise are still based on Java. Many developers that have mastered Java have also been reluctant to abandon it no matter how many other programming languages they may learn.

October 22, 2020
OpenTelemetry and the Future of Monitoring Instrumentation

OpenTelemetry is an open source project to provide a de facto standard trace API and a metric and log agent to end vendor lock-in

It’s no surprise that understanding the internal health of applications and systems is a high priority for developers and SREs. It’s a well-understood problem with dozens of vendors focused on helping companies diagnose application performance in real-time.

While the actual platform doing the metric, log or trace analysis tends to see most of the focus, a critical component of the data pipeline is, and has always been, the actual instrumentation. That includes APIs and SDKs for application traces as well as installable agents to gather metrics and logs. In the past, this has been a highly fragmented space with both open source and proprietary agents and SDKs. Most APM vendors have provided their own proprietary instrumentation libraries that, intentionally or not, create a vendor lock-in at a code-level that isn’t ideal for most organizations.

To address this challenge, an open source project was created with a highly aspirational goal of providing a de facto standard trace API along with a metric and log agent that would eliminate this fragmentation, create a unified standard and effectively end vendor lock-in.

The project, OpenTelemetry, was officially started in May 2019. It is a Cloud Native Computing Foundation sandbox project and came out of the merger of the OpenCensus and OpenTracing projects. They’ve since expanded their goal to include not only application tracing and code instrumentation, but also metric and log collection. They are essentially covering all the critical telemetry points in a system or application.

Imagine if, regardless of the platform you ultimately sent the data to, you could instrument your code with the same libraries, the same agents and a single set of binaries and libraries. It would dramatically simplify the deployment process, internal training and make it trivial to change or add analytics platforms as needed. This is the promise of the OpenTelemetry project.

A project like this doesn’t get far without significant support. This is where OpenTelemetry shows incredible promise to realize widespread adoption quickly. Nearly all the largest monitoring and cloud vendors including AWS, Google, Microsoft, Splunk, New Relic and many more have thrown their weight behind OpenTelemetry. They’re actively contributing and supporting the project. One by one, these organizations have come to the agreement that the telemetry component is not the differentiating feature. Instead, they are working together to provide a best-of-breed experience at the telemetry layer.

The project is in beta, and the potential is huge. OpenTelemetry modernizes every aspect of the telemetry system. It eliminates vendor lock-in. It will outperform the vast majority of existing agents. It simplifies installation and training by reducing dozens of agents and libraries to just one.

So where to get started?

It’s still early, but depending on your platform of choice, it may already support OpenTelemetry. If so, look into the existing libraries and test them out in your internal sandbox environments. You’re likely to find improvements over your existing telemetry tools, although there will certainly be gaps in things such as development language support and metric/log integrations. It should give you a sense of the scope, potential and maturity of the project and help to determine whether this is a viable solution for your next deployments.

The big question mark is whether this will live up to its potential. With the high-quality team assembled, the support of the community and industry and with a truly impactful mission, it will. Ideally, within a few years, this will be how we instrument and gather data from our applications regardless of where we send that data.

October 13, 2020
CloudBees Advances Expanding DevOps Agenda

CloudBees today at its online DevOps World 2020 conference announced it is embedding additional DevSecOps capabilities that span the application development and deployment process within the continuous integration/continuous delivery (CI/CD) platform based on the open source Jenkins projects.

In addition, CloudBees announced it will make generally available by the end of this year two additional modules for its Software Delivery Management (SDM) platform. Launched last year, the SDM platform provides a centralized mechanism for all stakeholders in the enterprise to track application development projects.

The first module available today unifies the management of feature flags across an entire application development project. The second creates a system of record for tracking progress by importing and normalizing all the data made available by the tools that make up a DevOps platform.

The DevSecOps offerings, meanwhile, provide IT teams with a hardened instance of the CloudBees CI offering, along with hardened audit-ready pipelines to ensure only immutable, approved components and environments are used without any drift or tampering. CloudBees is also adding support for role-based access controls based on fine-grained permissions as well as tighter integration with the feature-flagging capabilities of the CI/CD platform. If issues arise in a production environment, specific features can be pulled back with full traceability provided automatically.

Finally, CloudBees has integrated its CI/CD platform with security automation applications from Anchore, Alcide.io, FOSSA, CyberArk, Checkmarx, Contrast Security, Shiftleft.io, Snyk, RunSafe Security, Sonatype, WhiteSource Software, Synopsys and Zimperium.

CloudBees CEO Sacha Labourey said in the wake of the COVID-19 pandemic, it’s clear organizations are accelerating the development and deployment of more sensitive applications that drive digital business transformation initiatives. In fact, according to the results of a survey of 347 IT professionals conducted by Accelerated Strategies Group (ASG) on behalf of CloudBees published today, digital transformation objectives have somewhat or significantly increased in priority for nearly two-thirds (63%) of respondents.

Slightly more than half said their organizations have also placed higher priorities on increasing their DevOps initiatives (52%). Many are also now practicing daily standup meetings (56%), using cross-functional teams (46%) and automating tasks (43%).

Overall, 59% said their software teams have become somewhat or significantly more productive, with 43% reporting it is now easier to complete work tasks in a timely fashion. Nearly two-thirds (61%) said they are also finding it easier to work across time zones and, in some cases, even different continents (39%).

Labourey said that as digital business initiatives continue to gain steam, more organizations will be applying DevOps principles to the business itself. SDM platforms will play a critical role in enabling that transition by providing a common framework through which IT and business professionals will be able to collaborate more closely, said Labourey. The goal should not be to turn software development into a factory, but rather to prioritize projects and initiatives in a way that benefits the business.

Ideally, SDM also provides a framework through which IT teams can manage up at a time when many business leaders lack visibility into often complex DevOps processes that tend to have lots of dependencies, noted Labourey.

In the meantime, the divide between all the application owners, IT teams and security professionals that make up an IT community within an organization continues to narrow. The challenge now will be determining what to do next, once all those constituencies are finally seeing the same thing at the same time.

September 22, 2020
New Tracks = New Insights at DevOps World 2020

What’s new at DevOps World this year? Aside from the fact that this year’s event, happening today through Sept. 24, is being held virtually and is free of charge, DevOps World is also reorganizing the conference into three new tracks: Leadership, Community and CDF (Continuous Delivery Foundation).

Together, the three tracks illustrate the broad-ranging focus DevOps World strives to achieve. The conference aims to educate, entertain and elevate the DevOps community with informative, thought-provoking content. The Leadership track will spark conversations about DevOps now and into the future. The Community track focuses on how Jenkins devotees can better leverage the CI/CD resource inside their organizations. CDF track outlines best practices for CI/CD implementations.

DevOps World Tracks

Here are a few highlights from the new tracks at DevOps World 2020.

Leadership

Leaders aren’t born with their skills fully formed. They grow into their positions over time, growing stronger with training and experience along the way. This new track, chaired by independent analyst firm Accelerated Strategies Group (ASG), is designed to provide attendees access to strategic business thinking and deep technical expertise. The mix of presentations, panels, hands-on exercises and interaction with peers will equip a whole new generation of digital leaders.

What’s included in the Leadership track? Sessions will explore the ins and outs of everything from software quality to DevSecOps, from value stream thinking to the future of open source and DevOps in the enterprise. You’ll hear from CloudBees customers including T-Mobile and Fidelity Investments and thought leaders from ASG and IBM.

Do you want to know how to “Put Customers At The Center Of Software Delivery Management”? Forrester analyst Margo Visitacion has a session covering how leading companies are implementing this new, customer-centric operating model.

What can the U.S. military’s counterinsurgency doctrine teach you about advancing DevOps internally? Lindsay Holmwood, an engineering leader from Australia, will explain to attendees not only how to push DevOps forward but also how to fend off threats to its success. Naysayers will try to derail DevOps initiatives. Understanding the strategies and tactics used by the U.S. military to stop insurgencies can help you build a strong DevOps culture in your organization.

Community

This track pulls together the best elements of the movement that sparked the DevOps World conference into what it is today. It focuses on Jenkins, the world’s leading continuous integration and continuous delivery server, along with CI/CD tools, concepts and technologies that enthusiasts can use to do a better job delivering software.

Many sessions focus on Jenkins itself. CloudBees’ Oleg Nenashev’s breakout session, “Jenkins: Where It Is and Where It is Going,” provides an overview of the Jenkins configuration-as-code (CaC) ecosystem, Docker/Kubernetes support and environments as well as new Jenkins Pipeline features that create a shared vision for modern and cloud-friendly Jenkins. Other sessions include, “Machine Learning Plugins for Data Science in Jenkins,” “External Fingerprint Storage for Jenkins,” and “Jenkins UI Gets a Makeover.”

Practitioners from companies ranging in size from Wintellect to Salesforce will share other lessons of interest to the community. Ken Muse of Wintellect will give a talk on “Hyperscale Migrations – Tricks for Supporting Large Migrations with DevOps,” while Salesforce’s Scott Glaser will discuss his company’s “Culture of Collaboration and Sharing at Scale.” AKELESS’s Oded Hareven has pointers on how to “Streamline Your CI/CD Pipeline by Adopting a Just-in-Time Access Approach.” Splunk Inc.’s Chris Riley will outline ways to empower engineering teams in a breakout session, “How A Culture of ‘Be Yourself’ Generates More Revenue for the Business.”

CDF

This track is coordinated by the Continuous Delivery Foundation, a group created by the Linux Foundation to serve as a vendor-neutral home for many of the fastest-growing projects for continuous delivery. It pulls together nearly two dozen sessions on DevOps trends, technology resources such as Kubernetes and initiatives driven by the CDF itself.

One trend that’s catching on is the use of new organizational models to break down silos that can inhibit DevOps transformations. Capital Carbon Consulting’s Garima Bajpai discusses how a “Continuous Product Oriented Practice” can transform the qualitative way products are built and tracked into a quantitative way by identifying and measuring “value” in practice. The CDF’s Tracy Miranda and DeployHub’s Tracy Ragan present thoughts about how to bring more women into DevOps and help the open source community become more inclusive.

Kubernetes has become a de facto standard in container orchestration, so it’s probably no surprise there’s so much interest in the subject at DevOps World 2020. CircleCI’s Angel Rivera presents an overview of the containerization trend, arguing that a better understanding of runtime-related concepts will better prepare DevOps teams for the future. Cloud costs are on the agenda in Harness’ Tiffany Jachja’s session, which outlines ways attendees can optimize Kubernetes workloads.

What’s happening with CDF? A panel discussion with six CDF leaders will offer insights on the direction of the CDF, cover some of the strategic goals and review what’s next for the upcoming year.

To take advantage of these exciting new tracks, join us today!

September 22, 2020
Puppet Deepens Windows Integration

Puppet today announced it has extended support for Windows environments across its portfolio of IT automation tools by adding a Group Policy Migration capability that enables multiple Windows configurations to be tracked simultaneously and offering to assist organizations seeking to deploy the open source Chocolately package manager.

In addition, Puppet has extended its support for Microsoft tools by adding a Puppet PowerShell Desired State Configuration (DSC) Builder module, through which the Powershell tools Microsoft relies on to configure operating systems are directly exposed.

Finally, Puppet has also added a Puppet VSCode Extension that gives Windows administrators the ability to invoke the Puppet Domain Specific Language (DSL) to access capabilities such as Intellisense and error-checking using their existing user interface and text editor for Windows.

Puppet CTO Abby Kearns said interest in IT automation has significantly increased in the wake of the downturn brought on by the COVID-19 pandemic. While IT organizations that have been running instances of Linux have had access to a broad range of IT automation tools for years, Windows environments have not enjoyed the same level of parity even though half the workloads organizations are trying to manage today are based on Microsoft.net code, she said.

With more pressure to automate than ever, Puppet sees an opportunity to fill that void, Kearns added.

By relying on Puppet tools to automate both Linux and Windows platforms, an opportunity to reduce the total cost of IT also presents itself as organizations look to either limit or reduce IT headcount. Puppet claims there are already more than 40,000 organizations using its tools, including more than 80% of Global 5,000 organizations.

It’s not clear to what degree Microsoft is invested in IT automation tools, especially compared to IT vendors that make managing infrastructure as code both inside and out of the cloud their sole focus. However, since it launched Azure Microsoft has been investing in tools to manage infrastructure as code. The depth of the Puppet portfolio enables IT organizations to also apply IT automation to operating systems and now to applications via Chocolately.

Less clear is to what degree IT organizations may finally unify the management of Windows and Linux systems as part of an effort to reduce costs in the wake of the downturn. Most organizations continue to manage Windows and Linux systems in isolation. However, as IT organizations embrace IT automation more broadly, many are looking to standardize on a common set of best DevOps practices whenever possible.

In the meantime, the opportunity to extend IT automation into the midmarket remains significant. Most enterprise IT organizations have already embraced IT automation to a significant degree. Midmarket IT teams have generally not had the skills and resources required to manage infrastructure as code. However, as IT automation tools become more accessible to the average IT administrator, there may come a day soon when broad swaths of IT systems deployed by midmarket IT organizations will finally be automated.

September 21, 2020
Catchpoint Acquires Webpagetest.org Service

Catchpoint this week announced it has acquired Webpagetest.org as part of an effort to expand use of the open source web performance testing service.

Led by Patrick Meenan, the Webpagetest service is already relied on by more than 1 million users per month to instantly run tests of websites and applications from various locations, which are surfaced via a detailed audit that can be shared with developers.

Meenan has been personally supporting Webpagetest as a side project while working for a series of IT organizations and vendors starting with America Online (AOL). Now, with Catchpoint’s acquisition of Webpagetest.org, he will have a staff of contributors as a full-time employee.

Catchpoint CEO Mehdi Daoudi said the company will also incorporate Webpagetest into the tools it provides for monitoring digital experiences. The most appealing aspect of the Webpagetest service is that, rather than focusing on individual backend platforms, it tracks the actual performance that users experience across multiple types of browsers.

Daoudi said that as organizations accelerate the rate at which they are building and deploying digital business applications in the wake of the COVID-19 pandemic, there’s now more focus on web application performance than ever. The experience any user has when accessing these applications now impacts everything from their willingness to complete a transaction to their perception of the corporate brand, he noted.

In fact, a recent survey of 200 CIOs and 200 work-from-home managers conducted in July by ReRez Research on behalf of Catchpoint finds application performance is at the top of the list of concerns (66%), followed by network reliability, work-from-home technologies and security, at 63% each. Application performance is at the top of that list because both online revenue and employee productivity are now increasingly directly impacted by the overall application experience, said Daoudi.

There’s never been a shortage of tools for testing application performance. The issue is how those tools are often employed, he noted. Too many IT organizations are monitoring individual application components rather than the actual user experience.

Organizations that focus on end user experience in the age of digital business will, in time, separate themselves from rivals that merely make applications available online. A zealous appreciation for application performance needs to be ingrained in the culture of the organization for it to ultimately succeed, Daoudi said, noting that every employee, not just the DevOps teams, needs to be aware of the impact application performance has on the business.

That challenge is as IT environments become more complex and distributed, ensuring application performance becomes more challenging than ever. Cloud-native applications based on microservices may be easier to deploy and manage, but the dependencies between all those microservices can conspire in ways that degrade application performance over time. Determining the root cause of intermittent application performance issues can take weeks. The maddening thing is, many of the fixes required to address that issue once it’s discovered can be deployed in minutes.

Performance tools that surface those issues sooner than later are invaluable in a digital age, where time now quite literally is money.

September 18, 2020
Harmonizing Real-Time on Edge Platforms

Edge computing platforms sit at the boundary between the real-time world and the digital enterprise. Uninterrupted, uncorrupted data from the edge is a must if analytics in the cloud drive reliable business decisions. What’s the best approach: using Linux, an RTOS (real-time operating system), a hypervisor or something else for edge platforms? Let’s talk about harmonizing real-time on edge platforms and introduce a new foundation project in the Linux Foundation’s LF Edge family.

Real-Time and Deterministic

Real-time only means events are captured, tagged and processed as they unfold in the real world. That cadence can be seconds, milliseconds or microseconds depending on how many events and how often they occur and how tasks and algorithms stay in control. Three operating system characteristics determine how well real-time execution keeps pace: prioritization, interrupt response time and context switch time.

Prioritization ensures that tasks receive resources when needed. Tasks at a lower priority should pre-empt when a higher priority task requests service. A task refusing to yield to pre-emption can throw off the prioritization scheme, causing missed deadlines.

Interrupt response time measures the latency from the time an event requests service via a hardware interrupt to when the handler task executes. Real-time hardware platforms often place high-priority events on hardware interrupts for fastest pre-emption.

Context switch time is softer scheduling at work. It measures the time for a change in processing control from one execution entity to another, swapping out resources including registers, cache and memory as needed.

Deterministic means there are no surprises in behavior. The faster real-time events happen, and the harder a system works, the more likely non-determinism surfaces. One venture into virtual memory paging or a burst of networking activity can show up at an inopportune time, even though under most conditions real-time performance is adequate.

‘Hard’ vs. ‘Soft’ Real-Time

That’s why most people think of real-time as “hard” with a fast, deterministic operating system. RTOS platforms optimize the kernel and only build things in around it that are necessary, providing rigorous and sometimes mathematically proven scheduling for many tasks.

“Soft” real-time is more forgiving. On a fast processor, interrupt response time is tiny compared to the execution window. A system that isn’t deterministic can, if there is enough margin in context switch time, keep up with real-time events on a slower scale. Linux, while not deterministic, can handle some real-time needs through minimizing packages and tuning the kernel for better prioritization.

Linux also benefits from lighter context switching. Originally, execution entities in Linux were resource-heavy processes. Now, threads can handle tasks with far less switching overhead. POSIX threads standardized the approach with portability between Linux and RTOS environments.

How fast are threads? According to a recent synthetic thread benchmark by Eli Bendersky, switching on server-class platforms with enough memory, cache and storage can be in the range of 2 microseconds with 10,000 threads under ideal conditions. Eli pointed out his test environment has more RAM than many edge hardware platforms provide, but most edge applications won’t need that many threads, either.

So, the line for soft real-time has moved down into new territory, making the trade-off between hard and soft a bit blurrier.

Hypervisors and Orchestration

If your DevOps team commits to an RTOS, you may give up some familiar functionality and have a learning curve to deal with, but you get assured hard real-time performance. If you go with Linux, you get the tools and coding practices you’re used to, but soft real-time performance can be an issue as code piles up.

The leaders in RTOS platforms with enterprise networking noticed this, too. Recently they’ve turned to hypervisors in a bare-metal approach. A hypervisor can capture the benefits of environments working together, so teams are not forced into an either/or operating system choice. This is especially important if the edge application requires some form of certification, such as in medical, industrial or defense applications. Certified code can be compartmentalized on an RTOS, while Linux runs alongside.

Linux providers are turning more toward orchestration layers and containers. It’s an application focus speeding up deployment, with more scalability and less hardware dependence. It also brings the edge into the enterprise faster with the same application deployment strategy anywhere. More code means more determinism risk.

Convergence is a faint hope on this landscape with competing objectives, but a new approach offers a path for software reuse and rapid deployment.

Harmonizing the Edge

The Linux Foundation is active across the spectrum of IoT development, from the Zephyr open source RTOS to various embedded Linux projects. One of the newest projects in the LF Edge initiative is a foundation targeting a blend of deployment models in a consistent, open approach.

Project EVE (short for Edge Virtualization Engine) supports containers, clusters, virtual machines and unikernels simultaneously. It’s a slim bare-metal runtime offering hardware root of trust, device resource management and software update and patch functions. EVE can host any combination of apps in virtual machines or containers, including running an RTOS with its application in a virtual machine. It can even assign CPU and GPU cores to specific applications for enhanced determinism.

On the EVE road map is support for the ACRN hypervisor (another LF Edge project) and shrinking the EVE runtime for smaller embedded-class edge platforms. These efforts are essential where real-time requirements become harder and a hypervisor plus an RTOS makes sense. For softer real-time environments, EVE is adding support for Kubernetes K3S, Docker Compose, VPNs for public clouds and mesh networking for data flow between edge platforms.

While the project is still young, EVE holds great promise for DevOps teams working the edge. We’d look forward to seeing some benchmarking on a real-time scale; it likely handles soft real-time needs as well as Linux today. By harmonizing a range of diverse approaches into one foundation, EVE is creating a space for real-time on edge platforms no matter which development perspective teams come from.

September 18, 2020
Envoy Proxy Server Project Comes of Age

The momentum behind the open source Envoy proxy software project is increasing as the number of platforms it can be deployed on continues to expand.

Developed under the auspices of the Cloud Native Computing Foundation (CNCF), Envoy has most recently been extended to include support for Arm64 CI processors and Postgres databases.

Envoy provides a layer 7 edge and service proxy that can be employed to provide a layer of network abstraction for both cloud-native applications based on microservices and existing legacy applications based on virtual machines.

Initially developed by Lyft, there are now more than 180 IT vendors and more than 155 companies making contributions to Envoy. Matt Klein, a software engineer for Lyft who is also a maintainer for the Envoy project, said at that level it’s clear Envoy has achieved a critical level of mass. In fact, that level of support is why so many providers of service meshes have elected to deploy their frameworks on top of Envoy.

Klein conceded Envoy is a complex piece of software that requires a significant level of engineering expertise to master. However, as IT teams look to become more agile, the need for a layer of abstraction that reduces network complexity has become more apparent, especially as organizations move to embrace edge computing.

That agility now also can be extended to software platforms via plugins. In the case of Postgres, for example, Envoy can now decode and interpret the wire protocol format for Postgres to extract data and metrics from Postgres traffic. That capability will not only route database traffic more efficiently but it also offloads the management of that task from the Postgres server without any changes to the database platform required.

Envoy makes it possible to extend many of the infrastructure-as-code concepts first put forward to manage virtual machines in the cloud to networks. DevOps teams have been able to provision virtual machines in a matter of minutes for years. However, in on-premises IT environments it can take days, sometimes weeks, to provision networking resources. Envoy provides a layer of abstraction that makes it possible to manage network services at a higher level of abstraction.

Klein said in addition to making Envoy less complex, the team overseeing its development is working to add deeper support for WebAssembly, a platform that makes it easier to build applications on top of Envoy. The Envoy team also plans to tighten integration with multiple observability tools to enable DevOps processes to be applied more easily to networking services, added Klein.

As Envoy continues to evolve, the rate at which network operations and DevOps will converge is only going to accelerate. DevOps teams will be able to employ Envoy in concert with a service mesh to programmatically manage network services at will. It may take some time for roles within enterprise IT organizations to be redefined to reflect that new reality, but the days when networks were managed via command-line interfaces that had to separately accessed for each router, switch or appliance on the network are coming to a close. In its place, a new era of software-defined networking based on commodity processors is finally starting to rise.

August 27, 2020
GitLab Presses Case for DevSecOps Collaboration

GitLab today at its GitLab Commit Virtual event pledged to make securing its open source continuous integration/continuous delivery (CI/CD) platform a more collaborative effort.

Cindy Blake, senior security evangelist for GitLab, said the company is committed to sharing a road map developed in collaboration with DevOps teams that lets organizations see what cybersecurity issues will be addressed when.

Rather than merely sharing DevSecOps marketing collateral, Blake said the goal is to encourage engineers to participate in a feedback loop that will help prioritize what issues need to be addressed across a shared DevSecOps agenda.

Most of that effort thus far has focused on making tools for scanning for vulnerabilities a natural extension of a DevOps workflow by embedding them into GitLab Core. Most recently, GitLab acquired Peach Tech, a provider of protocol fuzz testing and dynamic application security testing (DAST) API testing tools, and Fuzzit, a continuous fuzz testing tool. The company also moved to make its CI/CD platform available as a set of hardened Docker container images.

The next priority is to enable organizations to construct DevSecOps workflows spanning DevOps and cybersecurity teams, said Blake, noting it will be easier to construct workflows at scale using a CI/CD platform that tightly integrates all the tools required.

In time, GitLab plans to extend those efforts to include machine learning algorithms that will be trained to identify and remediate cybersecurity issues, she added.

As a founding member of the Open Source Security Foundation, GitLab is also committed to working with the rest of the open source community to better secure software, Blake said.

There’s naturally a lot more focus these days on application security. A recent Global DevSecOps Survey conducted by GitLab finds developers are exercising more control over security—more than 25% of developers reported feeling solely responsible for security, compared to 33% of security team members who say they own security. A total of 29% of respondents said they believe everyone should be responsible for security.

However, cybercriminals are also making a concerted effort to compromise software supply chains by inserting malware into DevOps workflows constructed on CI/CD platforms. The challenge with securing open source platforms, of course, is the code employed to build these platforms is accessible to anyone including, of course, cybercriminals.

It’s too early to say to what degree cybersecurity concerns are influencing the selection of CI/CD platforms. However, as cybersecurity professionals become more involved in application development, it’s only a matter of time before issues involving the security of the platforms used to build applications become a higher concern.

In the meantime, the line is blurring between application development and cybersecurity. The challenge for each organization will be determining just how far left they want to shift responsibility for cybersecurity to the individuals who build their applications versus a cybersecurity team that has a less vested interest in accelerating the rate at which code is promoted into a production environment.

August 26, 2020