devopsy.kinsta.cloud

Author: Mike Vizard

Perforce Acquires 21 Labs to Advance Mobile App Testing

Perforce Software announced today is has acquired 21 Labs, a provider of a suite of tools that leverages artificial intelligence (AI) to provide a no-code approach to testing mobile applications. Terms of the deal were not disclosed.

Tim Russell, chief product officer for Perforce Software, said the 21 Labs tools complement a set of similar AI-based tools for testing web applications that the company makes available via its Perfecto portfolio. The company also makes available Virtual Devices, a service that makes it feasible to test mobile computing devices at scale using Apple iOS and Google Android emulation software. Perforce acquired Perfecto in 2018.

21 Labs developed a test automation platform that uses algorithms to observe interactions within a mobile application that are then used to generate tests. That approach sharply reduces the need for a developer or professional application developer to employ scripts to generate those tests, said Russell.

The next step will be to integrate 21 Labs with the test recommendation engine that Perforce already makes available via its Perfecto testing platform, added Russell.

As the rate at which applications are developed accelerates, Russell noted, application testing has become a major bottleneck. The shift toward scriptless approaches to application testing will make it possible for organizations to quickly vet applications to ensure quality without having to slow down the rate at which they are being deployed. That’s especially critical at a time when mobile applications are often at the core of any digital business transformation initiative.

It’s not clear to what degree application testing will be automated, but it’s apparent that as AI becomes more advanced, the need for many of the routine tests conducted by professional developers today will be eliminated. In theory, that should afford professional developers more time to conduct complex tests, while more routine tests are automatically run as an extension of a DevOps workflow.

Application testing, of course, has always been the first thing to suffer whenever developers are pressed for time to make a delivery deadline. However, the sooner any issue is discovered, the less expensive it generally is to fix. At the same time, end users are not generally patient when it comes to mobile applications that don’t work as advertised. External-facing mobile applications, especially, need to provide a compelling experience the first time they are fired up. At the same time, internal digital business processes that are accessed by end users and business partners now need to be accessible from anywhere.

Arguably, it’s harder to provide a great application experience on mobile computing devices than it is on a desktop platform, which means there should be even more testing of mobile applications before they are deployed in a production environment. The days when developers experimented on end users by rolling out minimally viable applications are coming to an end, as end user expectations continue to rise. DevOps teams that don’t adjust to that new reality may soon discover just how critical testing has become to their own job security.

May 17, 2021
WhiteHat Security Allies With Bit Discovery on Vulnerability Intelligence

WhiteHat Security, a subsidiary of NTT, has announced an integration with Bit Discovery to make it easier for developers and cybersecurity teams to discover the extent to which the attack surface they need to defend might be impacted by a vulnerability.

Bit Discovery maintains a database made up of more than 5 billion internet-connected assets and hundreds of third-party resources to automatically generate an inventory of the assets that make up an organization’s attack surface. Bit Discovery continuously scans internet-connected assets to monitor new domains, phishing sites and connected devices to provide cybersecurity teams with a deeper understanding of the threats their organization faces.

WhiteHat Security integrated its application security platform for scanning for application vulnerabilities with the Bit Discovery database via application programming interfaces (APIs) to make it possible to search that database via a dashboard. All vulnerabilities surfaced via that dashboard are verified by WhiteHat’s Threat Research Center, a team of application security experts, to reduce any potential alerts that might actually be false positives.

Craig Hinkley, WhiteHat Security CEO, said the goal is to make it simpler for organizations to shift more responsibility for applications security left toward developers in a way that doesn’t result in them chasing their tail every time a new vulnerability is discovered. Instead, a virtuous cycle of workflows can be created that enables developers and cybersecurity teams to collaborate more effectively, Hinkley added.

Ultimately, the goal is to make scanning for application vulnerabilities a more natural extension of any quality assurance process. The simple fact that there is so much focus today on DevSecOps best practices only highlights how much work there is to be done before security becomes just another routine gate within an automated DevOps workflow, noted Hinkley.

In the meantime, unfortunately, cybersecurity teams are finding it increasingly difficult to keep pace with the rate at which applications are being developed and updated. The Bit Discovery database integration provides a way for cybersecurity teams to have more relevant conversations with developers about which vulnerabilities need to be prioritized based on the actual threat they represent to the organization. In the absence of that integration, security becomes a losing battle, simply because developers and cybersecurity teams both are overwhelmed by the volume of vulnerabilities that would otherwise need to be investigated on their own without any assistance.

The integration between WhiteHat Security and Bit Discovery comes at a time when the focus on software supply chains has increased significantly in the wake of a series of recent high-profile breaches. More attention is now being paid to preventing malware from finding its way into the application development process. The challenge is finding a way to provide developers with actionable insights into risks that need to be mitigated before and after an application is deployed in a production environment. After all, vulnerabilities are just as often discovered after an application is deployed as they are before.

It may be a while before most organizations are able to implement a robust set of DevSecOps best practices, but the more reliable the vulnerability intelligence being provided to developers, the more likely it becomes they will act on it before a cybercriminal exploits it.

May 17, 2021
Appian Extends Low-Code Approach to Aggregating Data

At an online Appian World conference, Appian today unveiled an update to its low-code platform that adds a set of visual tools that enables developers to aggregate data within an application with the help of a database administrator (DBA) or data engineering team.

Malcolm Ross, vice president of product at Appian, said one of the things that holds up application development the most is the need to wait for data to be massaged in a way that makes sense within the context of an application. The latest edition of the Appian Low-code Automation Platform aggregates multiple data sources into a single data vector columnar data store that resides within the platform.

In effect, Appian is applying a low-code approach to data engineering processes that, today, require dedicated expertise to set up and manage.

The latest version of Appian also adds an Intelligent Document Processing (IDP) tool enabled by optical character recognitions and AI models, application design guidance, developer collaboration tools and enhanced DevSecOps capabilities that enable software packages to move between development, test and production environments with a single click in a way that is governed by the Appian platform.

Appian is also extending robotic processing automation (RPA) tools to centrally manage bots within an application to Windows platforms.

Finally, Appian is making a formal, less-robust community edition of its platform available to a limited number of developers as part of an effort to expand the overall size of its developer community.

Ross said as both professional and so-called citizen developers embrace low-code applications it’s critical to have guardrails in place that ensure applications are deployed in a way that adheres to DevSecOps best practices. The Appian Low-code Automation Platform includes the software development life cycle (SDLC) tools needed to set those guardrails regardless of the level of experience any developer has, noted Ross. In fact, Appian guarantees, for a flat services fee, that an organization’s first project will go live in just eight weeks.

In general, Ross said the number of individuals capable of building applications continues to expand as more “business technologists” learn how to build applications. Previously, these individuals might have used 4GL tools or macros to build applications based on Microsoft Excel spreadsheets. The Appian low-code platform makes it possible for this class of power end users to build more robust enterprise applications using a set of tools that compare their efforts to known best practices and offer suggestions to make improvements.

At the same time, professional developers are opting to build applications using low-code tools to reduce the size of application development backlogs. Not every application that needs to be developed requires developers to employ procedural code.

The net effect is more applications than ever are being simultaneously built and delivered at a time when organizations are launching multiple digital business transformation initiatives.

It may be a while before the effects of low-code platforms are fully realized within IT environments. However, the one thing that is certain is IT teams of all sizes should expect to be managing an exponentially higher number of applications.

May 11, 2021
OpsRamp Extends AIOps Reach to Embrace Prometheus

OpsRamp today announced it has added the ability to ingest data captured via the open source Prometheus monitoring software into its namesake IT operations platform infused with machine learning algorithms.

In addition, the company is adding support for customizable dashboards using the Prometheus Query Language to create visualizations of hybrid infrastructure performance. Also, a set of available tools makes it easier for IT teams to discover when workloads have been added to a cloud. These capabilities became available with the Spring 2021 Release of OpsRamp’s IT operations platform infused with artificial intelligence (AIOps).

Finally, OpsRamp has also created an alert definition model that streamlines the alerts generated by the AIOps platform and its integrations for the Microsoft Azure cloud and the Cisco HyperFlex platform deployed in on-premises IT environments.

Deepak Jannu, senior director of product marketing for OpsRamp, said that as IT becomes more complex to manage, many IT teams are starting to realize that managing these environments without help from an AIOps platform won’t be possible. As such, much of the fear and loathing that was initially associated with AIOps is giving way to acceptance as IT teams start to realize these platforms augment their capabilities versus replace them, added Jannu.

Much of the complexity is being driven by the rise of cloud-native, microservices-based applications that tend to have lots of dependencies. OpsRamp has primarily been encountering Prometheus in these environments. However, Prometheus can also collect data from legacy applications, as well, so over time, the expectation is the open source monitoring tool will be employed more widely.

As AIOps platforms become more widely employed, there is no doubt job roles within IT organizations will change. As lower-level tasks become automated, IT teams should have more time to spend resolving complex issues once the AIOps platform has identified a baseline for how applications should optimally behave. In many cases, the need for personnel to perform rote tasks that no one in IT wants to actually do will be eliminated altogether.

DevOps teams that are committed to automating IT tasks will naturally be among the first to employ an AIOps platform to augment the capabilities of site reliability engineers (SREs) that are often challenging to hire and retain.

In general, Jannu noted that the observability that many IT organizations now demand can’t be achieved by simply collecting more metrics. IT organizations will need to invest in an AIOps platform capable of making sense of all the instrumentation data being collected. The challenge now will be defining the right division of labor between man and machine in an IT environment, Jannu added.

At this juncture, AIOps, to one degree or another, is inevitable. In many cases, machine learning algorithms are already being employed to optimize applications or a specific class of hardware. The decision IT teams will ultimately need to make is determining whether to replace their existing IT operations platform or wait for the provider of their existing platform to get around to adding AI capabilities.

May 11, 2021
CircleCI Acquires Vamp to Apply AI to CD

CircleCI today announced it has acquired Vamp, a provider of an orchestration platform for automating application releases. CircleCI at the same time revealed it has garnered an additional $100 million in financing to attain a $1.7 billion valuation.

Jim Rose, CircleCI CEO, said Vamp will play a significant role in advancing the capabilities of the CircleCI continuous integration/continuous delivery (CI/CD) platform at a time when the rate at which applications are being deployed continues to steadily increase. In fact, CircleCI claims its platform is now being employed to process 2.5 million jobs processed per day.

The Vamp platform makes it easier to construct and orchestrate a multi-stage release cycle involving, for example, canary releases of an application. It then employs machine learning algorithms to validate the impact, performance and quality of releases based on the service level objectives defined by an IT team. If anything is determined to be amiss, the platform will then automatically roll back a release.

While many organizations have successfully implemented CI best practices, the ability to continuously deploy application releases has proven to be a more elusive goal. Each platform on which an application is deployed has unique attributes. The addition of machine learning algorithms that can provide continuous validation of deployments should help drive a sea change in the way applications are being developed and deployed to improve the overall state of CD, noted Rose.

Those machine learning algorithms, along with other forms of artificial intelligence (AI) won’t replace the need for DevOps teams as much augment them at a time when IT environments, thanks in part to the rise of microservices, are becoming more complex. In fact, with the arrival of containers, Kubernetes and serverless computing frameworks, it’s becoming next to impossible to manage an IT environment without the use of AI.

Rose added that, as the line between pre-production and production environments continues to blur, it’s critical for IT teams to more closely align CI and CD workflows. Rather than isolating CD from CI within the context of a GitOps workflow, the goal should be to integrate those processes more closely. It is becoming more apparent that the elements of what constitutes an application are more empirical thanks to, for example, configuration files that are now stored in a Git repository along with application code. That tighter integration ultimately makes it easier to test and validate applications before they are deployed. Metadata that can be analyzed to optimize application development and deployment can also be more easily captured as part of a continuous feedback loop.

That approach is more critical than ever, because the days of the “Big Bang” deployment of an application update are over. In its place are now a steady stream of updates that deliver new capabilities, which need to be continuously vetted, noted Rose.

Each successive era of DevOps is, naturally, going to be more automated than the previous one. The challenge is recognizing when those new eras arrive before rivals gain an advantage in speed and agility that becomes too big to overcome.

May 11, 2021
Linux Foundation Embraces Lightweight API Gateway

The Linux Foundation announced today that an open source application programming interface (API) gateway known as KrakenD has become the latest project it will oversee.

Rechristened the Lura Project, the lightweight stateless API gateway was originally developed by KrakenD, which plans to continue to offer an enterprise edition of the platform.

KrakenD CEO Albert Lombarte said the API gateway it developed provides a more streamlined approach to managing APIs that doesn’t require an organization to install a separate proxy or database. All the reverse proxy capabilities most organizations require is already embedded within the API gateway. That approach also makes it easier for IT teams to decouple back-end server applications from client applications because now an API can be more easily updated without impacting client applications, Lombarte added.

Organizations that already contribute to the ongoing development of the API gateway include 99P, Hepsiburada, Openroom, Postman and Stayforlong. The API gateway also is embedded in a variety of other platforms, said Lombarte; he noted that the number of servers running the gateway now exceeds more than one million per month.

API gateways have been available in various forms for years, but with the rise of microservices the number of APIs organizations need to deploy and manage continues to rise steadily. Each microservice added to an environment adds yet another API. At a time when microservices-based applications are at the core of many digital business transformation initiatives, IT teams must employ a more flexible approach to managing the interactions between APIs.

Thanks in part to the role they play in enabling digital business transformations, APIs are also being targeted more frequently by cybercriminals that have come to view them as an efficient way to exfiltrate data or inject malware into an application environment.

It’s not clear what impact the Linux Foundation’s support might have on organizations’ selection of one API gateway versus another. In fact, many organizations are still trying to navigate when to employ an API gateway versus relying on proxy software to manage a handful of APIs versus a service mesh to manage thousands. The issue comes down to not just how many APIs there are, but also the overall weight of the platform being employed, said Lombarte. A service mesh typically requires more time and effort to deploy and manage, he noted.

As the number of APIs in an IT environment continues to increase, there may come a day when organizations will have to revisit how APIs are created and maintained. All too often, APIs are still created as an afterthought once an application is created. They tend to be brittle and, as a consequence, still break often. A more holistic approach to managing APIs based on a set of DevOps best practices is required. The challenge is getting DevOps teams to prioritize APIs when there are so many other issues that require their attention. Nevertheless, it’s become apparent APIs are, in many cases, becoming too strategic to the business for their management to be left in the hands of a single development team.

May 11, 2021
ServiceNow Acquires Lightstep to Gain Observability Platform

ServiceNow today announced it has acquired Lightstep as part of an effort to add an observability platform to its IT operations portfolio. Terms of the deal, expected to close this quarter, were not disclosed.

The Change Intelligence observability platform from Lightstep is based on a time-series database that is capable of processing a trillion events each day. It was built by the same team that created Monarch, the globally distributed in-memory time series database system that Google employs to monitor its applications and systems. Lightstep team members also co-created both the OpenTracing and OpenTelemetry open source projects for collecting traces, metrics and logs.

Pablo Stern, senior vice president and general manager for IT workflow products at ServiceNow, said ServiceNow will employ Lightstep to enable IT organizations to observe both monolithic and microservices-based applications as IT operations and DevOps teams continue to collaborate more closely with one another. The goal is to provide a single pane of glass through which IT teams can observe their entire IT environment via the Now platform that ServiceNow makes available via a software-as-service (SaaS) delivery model.

That approach will enable DevOps teams to programmatically manage IT environments alongside IT administrators employing graphical tools, noted Stern.

Lightstep CEO Ben Sigelman added that, in time, the number of users of that observability will only increase as more application stakeholders start to track software development projects that are at the heart of various digital business transformation initiatives.

Thanks mainly to the rise of microservices that are now being deployed across highly distributed computing environments, it’s become more difficult to discover the root cause of an IT issue. Microservices-based applications have lots of dependencies that, while serving to make the overall application more resilient, conspire to make it difficult to determine which specific microservices might be adversely impacting the performance of an application.

The time-series database that lies underneath the Change Intelligence observability platform makes it feasible to go beyond logs and metrics to process distributed traces that provide more insight into application behavior. Ultimately, the goal is to enable organizations to innovate faster with a higher degree of confidence in the IT environment, said Sigelman.

As far as the march to observability in the enterprise is concerned, it’s still early days. While observability may be a core tenet of any DevOps practice, most IT organizations rely on myriad tools that surface data the teams then attempt to correlate. Whenever there is a major issue, those IT teams typically convene in a “war room” to compare reports generated by the various IT monitoring tools employed. Observability platforms that collect and process distributed traces used to instrument an application using a consistent set of specifications, frameworks and libraries. The OpenTracing and OpenTelemetry tools are significant because they provide an open source approach to instrumenting applications that reduces the total cost of observability. In comparison, many legacy application performance management (APM) platforms employ proprietary agents to instrument applications.

Naturally, each IT organizations will be moving down the path toward an observability platform that provides greater application context. At this point, however, it’s not clear to what degree observability platforms will need to be installed before making the transition to microservices versus being a capability that only gets appreciated once the trouble with microservices finally becomes apparent.

May 10, 2021
Trend Micro Allies With Snyk to Advance DevSecOps

Trend Micro and Snyk unveiled today a software-as-a-service (SaaS) platform the two companies have jointly developed to identify vulnerabilities in open source code.

The Trend Micro Cloud One – Open Source Security by Snyk platform enables DevOps teams to both identify vulnerabilities and licensing issues to better monitor, prioritize and share information about risk and exposure rates within application development projects.

Trend Micro COO Kevin Simzer said the SaaS platform provides a unified approach that combines six services within a single subscription to enable organizations to better protect a software supply chain.

Snyk CTO Geva Solomonovich said that while the relationship between the two companies is not exclusive an alliance with Trend Micro will enable organizations to employ Snyk tools and the open source vulnerability database to implement a set of best DevSecOps practices within their organization.

Simzer said Trend Micro is also including automation tools that make it easier for cybersecurity teams, along with the rest of the IT organization, to discover what application development projects are underway with their organization. The goal is to provide cybersecurity teams and DevOps teams with a set of tools that provides them with a common language through which they can collaboratively address vulnerabilities.

Trend Micro claims approximately eight hours can be saved per vulnerability through automation and early discovery of the versions of open source code being employed. Roughly 80% of all application code in use today is derived from open source software, noted Trend Micro. That translates to saving more than 650 hours of development time per application, according to the company.

Snyk, meanwhile, claimed to have observed a 2.5x increase in the number of open source vulnerabilities it has discovered over the past three years. In the wake of some recent high-profile breaches the security of software supply chains has become a greater area of focus, with many organizations now implementing a more rigorous review of all the code they employ and any associated dependencies.

It’s still early days as far as adoption of DevSecOps best practices are concerned within most organizations. Responsibility for application security is clearly shifting further left toward developers. As part of that effort, cybersecurity teams are trying to adjust their processes in a way that enables them to vet the efforts of developers without slowing down the rate at which applications are developed. That’s becoming more challenging as the rate at which applications are being developed continues to increase.

The conundrum organizations are trying to navigate is that every minute a developer spends on security is, theoretically, one less minute they have to write applications or business logic. At the same time, however, if vulnerabilities are not addressed early in the development cycle, they may take longer and cost more to fix, especially if they are discovered after an application is deployed in a production environment.

Of course, there may come a day a when application security is just another set of steps to be performed within a quality assurance process. In the meantime, however, a lot more attention is being paid to application security today, in part, to make up for the many sins of the past.

May 10, 2021
Survey Illuminates Long Dark Road to Hybrid Cloud

While hybrid cloud computing will play a critical role in enabling digital business transformation it turns out most IT organizations lack the skills and technologies required to achieve it.

A survey of 108 IT executives conducted by the research firm Pulse on behalf of CloudBolt Software , a provider of a cloud management platform, finds 94% believe hybrid cloud computing will be critical to digital transformation. A total of 92% also noted that empowering end users through self-service IT is a digital transformation accelerator, while 99% said successful automation depends on streamlining integrations.

However, 78% of respondents said they lack the visibility necessary to optimize cloud deployments. More than half (56%) believe that their self-service offerings fail to meet user needs and more than three-quarters (76%) still rely on some form of custom coding when it comes to integration.

In terms of the capabilities respondents are looking to acquire, 80% noted they want real-time insight into cost optimization and security remediation. A total of 71% want self-service to be easy for end users, while 62% want an approach to integration that doesn’t require deep domain knowledge or special expertise.

CloudBolt CEO Jeff Kukowski said the biggest immediate obstacle when it comes to hybrid cloud computing is the lack of visibility IT organizations have into their existing IT environments. IT teams are not going to be able to unify the management of distributed IT environments without it, much less achieve anything approach observability, noted Kukowski.

Pressure on IT organizations to accelerate application development while simultaneously reducing costs rose as the economic downturn brought on by the COVID-19 pandemic began. The challenge is that each new platform adopted by an organization tends to increase the total cost of IT as organization acquire tools and hire specialists to manage those platforms. It’s not likely all those platforms will be managed via a single console, but Kukowski said it’s clear IT organizations are moving toward a more federated approach that will enable them to centralize the management of IT.

In the meantime, Kukowski noted organization are collectively wasting billions of dollars on IT platforms that are employed inefficiently. Most of that waste stems from either overprovisioning of IT infrastructure, or simply because someone didn’t think to turn off a virtual machine left running on a cloud service.

It may be a while before IT organizations are able to strike that perfect balance between flexibility and control that currently remains elusive. However, organizations that focus on improving a few small things today will find themselves in a much better position to manage what is rapidly becoming an over extended enterprise tomorrow. Right now, Kukowski estimated only about a quarter of IT organizations have the sophistication required to manage a hybrid cloud computing environment.

There’s no doubt the management of IT will become more automated in the months and years ahead. The challenge organizations face today is determining what tasks either can or soon will be automated. Once that’s determined, it becomes a lot easier to identify what DevOps skills those IT teams will require tomorrow.

May 6, 2021
Splunk Unveils Unified Observability Platform

Splunk today unveiled an observability platform delivered as a cloud service that captures all metrics, traces and logs in real-time without relying on sampling.

Spiros Xanthos, vice president of product management for observability and IT Ops at Splunk, said Splunk Observability Cloud collects all that data via a single agent based on open source OpenTelemetry software to instrument an application.

Historically, application performance management platforms (APMs) have only analyzed samples of data to identify a potential anomaly. Splunk Observability Cloud analyzes all the streaming data captured by its agent software to enable IT teams to pinpoint the root cause of an issue faster, said Xanthos.

As a result, Xanthos said the signal-to-noise ratio dramatically improves compared to legacy monitoring platforms. It will then be up to each IT team to decide how much of that data they want to store after it’s been analyzed, added Xanthos. Pricing for Splunk Observability Cloud is based on the number of hosts employed rather than the amount of data collected.

The Splunk Observability Cloud unifies a Splunk analytics portfolio by including capabilities provided via Splunk Log Observer, Splunk Real User Monitoring (RUM), Splunk Synthetic Monitoring, Splunk Infrastructure Monitoring, Splunk APM and Splunk On-Call. Splunk Synthetics Monitoring is based on technology Splunk gained with the acquisition of Rigor last year.

The Splunk Observability Cloud is one of the first such platforms to rely solely on OpenTelemetry agent software being developed under the auspices of the Cloud Native Computing Foundation (CNCF) to collect data. That project not only reduces the cost of instrumenting applications, it also reduces the number of agents an IT team might have to deploy to collect metrics, traces and log data.

Many rival observability platforms can collect data from OpenTelemetry agent software, but also tend to encourage DevOps teams to deploy proprietary agent software that they claim is easier to install and collects data at a more granular level. Splunk has decided OpenTelemetry has matured to the point where those proprietary agents are no longer required, said Xanthos.

The rise of observability platforms presents an opportunity for IT teams to rationalize a raft of monitoring tools that often surface conflicting analyses. IT teams today spend an inordinate amount of time convening so-called “war room” meetings to determine the actual source of an issue. Those meetings can go on for hours, only to eventually discover an issue that only takes a few minutes to fix.

While observability has always been a core DevOps principle, the level of visibility most IT teams have into their application environments is limited. There is no shortage of monitoring tools, but because there is no unified approach to observing an IT environment, there simply isn’t enough context being provided. As IT environments become more complex – thanks primarily to the rise of microservices-based applications that have lots of dependencies – that lack of context is rapidly becoming a major issue. A microservices-based application is designed to degrade gracefully if one microservice suddenly become unavailable. The issue is that it can take a very long time to discover the root cause of performance degradation.

It may be a while before most IT teams achieve true observability, but once they do, many will wonder how they previously managed IT without it.

May 5, 2021