devopsy.kinsta.cloud

Author: Mike Vizard

NodeSource, AppDynamics Ally for Node.js DevOps

The prevalence of Node.js as the most widely deployed instance of JavaScript is putting a lot of pressure on DevOps teams to find ways to automate the management of Node.js applications. To provide the visibility needed to achieve that goal, NodeSource has announced it has integrated its N|Solid development environment with the application monitoring tools developed by AppDynamics.

Pravin Halady, product manager for NodeSource, says one of the things that differentiates N|Solid, which is based on Node.js, most is the amount of telemetry data NodeSource is able to collect from Node.js applications. The alliance with AppDynamics provides a mechanism for DevOps teams to now tap into that data to better manage those applications alongside the panoply of existing applications they need to support and manage, says Halady.

AppDynamics already supports Node.js. But Halady says the alliance with NodeSource provides deeper visibility into, for example, asynchronous activity. Advanced metrics can be used to augment the basic Node.js metrics that AppDynamics currently provides to better ensure service level agreements (SLAs) are both achieved and maintained, Halady says.

Adoption of Node,js exploded across the enterprise once it became feasible several years ago to deploy JavaScript code on both the client and the server. That capability in many ways fulfilled the early promise of Java in the eyes of many developers. But DevOps teams have been slow to include support for JavaScript applications within their DevOps processes, mainly because of a lack of visibility. Now that the base of Node.js applications in the enterprise has reached a level of critical mass, tools vendors such as AppDynamics are looking to provide that visibility by expanding the scope of support they provide for Node.js.

Halady says NodeSource chose to partner with AppDynamics in part because in the wake of AppDynamics being acquired by Cisco Systems, it’s clear that NodeSource needs to participate in the end-to-end visibility platforms that larger companies are starting to build. Cisco has already made significant investments in a Cisco Tetration analytics application in addition to committing to imbuing the IT infrastructure environment it provides with machine-learning algorithms. That data will then be combined with data about applications collected by AppDynamics to identify the source of any potential performance issue faster. NodeSource, adds Halady, will look to foster similar agreement with other providers of application monitoring tools.

In the meantime, DevOps team can take heart in the fact the Node.js Foundation is making providing visibility into DevOps environments a higher priority. The number of Node.js applications in the enterprise is only expected to increase. There’s a high correlation between usage of Node.js and containers that collectively are being used to facilitate the development of microservices. As those microservices bgin to replace monolithic applications developed originally developed in languages such as Java, the size and scope of the DevOps challenge exponentially increases.

Application development in the enterprise is going to be polyglot well into the next decade. Today there are more programming languages being actively used in the enterprise than any time in history. In fact, the days when IT operations could mandate what technologies developers can employ are long over.

— Mike Vizard

September 6, 2017
VMware Cloud on AWS Will Bring DevOps Issues to a Head

Now that instances of VMware are starting to become available on Amazon Web Services (AWS), a DevOps ecosystem is starting to emerge around the management of applications workloads moving into and out of AWS.

Among the more than 4,500 applications that VMware claims already support VMware Cloud on AWS are those from Blue Medora, CA Technologies, Chef, Cloudbees, Dynatrace, Jfrog, Puppet, SaltStack and the Ansible arm of Red Hat.

VMware CEO Pat Gelsinger told attendees at the recent VMworld 2017 conference that VMware and AWS are just now scratching the surface of what they plan to accomplish together. AWS CEO Andy Jassy added that, unlike other alliances, which consisted mainly of marketing, engineers at both companies are working closely together to deliver VMware globally across all AWS availability zones. At present, VMware Cloud on AWS is available in the AWS availability zone hosted in Portland, Oregon. The service soon will be extended to the east coast of the United States, followed by rest of the world over the course of 2018.

The availability of VMware Cloud on AWS should force several DevOps issues within enterprise IT organizations that have deployed the VMware stack of software mainly on-premises. Many of those organizations have also been employing public clouds such as AWS. But because those environments are based on different hypervisors, it was never easy to migrate workloads between the two platforms. There are plenty of options for running VMware software in the cloud, but because AWS is the largest provider of public cloud services, there are many customers who want to seamlessly move workloads between their on-premises IT environments and AWS via a common stack of VMware software.

In many organizations the desire to shift applications workloads into AWS will bring many DevOps issues to a head, as VMware and AWS use has occurred in independently of each other. There could be many workloads running on each platform, but for all intents and purposes, IT organizations have managed both platforms in isolation. Now that VMware is becoming available on AWS, all the challenges associated with moving workloads between a public and private cloud come to the fore.

Most of those DevOps issues will be focused first on moving some classes of workloads running on VMware into AWS. Application workloads that don’t need to run constantly will be among the primary candidates followed by a slew of data protection applications that will enable IT organizations to back up and archive massive amounts of data inexpensively. Next up will be applications that need to be distributed globally to increase availability.

At the same time, a lot of VMware shops will be moving application development into the cloud. But just because an application is developed in the cloud doesn’t mean it will stay there. Many IT organizations have already discovered that when it comes to long-running applications, the total cost of deploying them on-premises is less than deploying them on a public cloud. The DevOps tasks ahead, of course, will start with figuring out which VMware-based applications make the best candidates from a financial and availability perspective to be deployed on AWS.

— Mike Vizard

September 5, 2017
VMware Aims to Advance Application Security

Most of the tasks associated with maintaining IT security assume that IT security professionals should discover and neutralize all potential threats to an application. At the VMworld 2017 conference this week, VMware is seeking to turn that notion of IT security on its head by focusing the efforts of IT security professionals on the known good.

Tom Corn, senior vice president for security products at ware, says instead of chasing all the potential bad things that could adversely impact an application, VMware App Defense employs machine-learning algorithms and data VMware collects from its management console and the host on which the applications are running to determine what elements make up an application workload. Those elements are compared to the manifest that was created when the workload was first deployed and, if they have been altered, VMware App Defense will invoke a preloaded library of incident response routines to enforce policy using VMware network virtualization software. Data collected from the host by VMware App Defense is done by inserting code into the kernel rather than deploying agent software.

Corn says VMware App Defense uniquely provides DevSecOps teams with access to reliable telemetry into the guest operating system environments running on top of VMware. IBM will feed the telemetry data gathered by VMware App Defense into its portfolio of security analytics software, and eventually that data will be fed into the IBM Watson cognitive computing platform to generate application security recommendations. Other vendors pledging support for VMware App Defense include VMware sister companies RSA and SecureWorks, as well as Carbon Black and Puppet.

Longer term, Corn says VMware expects to also be able to apply this capability to any production environment attached to a VMware NSX virtual network overlay.

As potentially compelling as VMware App Defense might be, Corn notes that IT organizations should not attempt to boil the IT security ocean. Rather, they should concentrate on new applications currently under development. Otherwise, managing security policies across hundreds of applications made up on thousands of microservices and communicating over microsegmented network overlays might be too complex to manage.

VMware has been making a case for some time now to employ NSX to take advantage of microsegmentation to limit the flow of east-west traffic in a data center. The idea is that if one node gets compromised by malware, it can only share that malware with nodes participating on the same microsegment of virtual network. VMware is now expanding on that idea in a way that also provides more visibility all the way up the application stack.

Priced at $500 per CPU per year, VMware App Defense will induce more IT security professionals to recommend deploying its stack over any other available option, VMware hopes. The degree to which IT operations teams and cybersecurity professionals can influence developers in that direction remains to be seen. But VMware App Defense does fulfill its promise, it’s more than likely developers will appreciate the not having to spend as much time defending existing applications instead of writing new ones.

— Mike Vizard

August 29, 2017
VMware Ties Desktop Virtualization to DevOps Productivity

It’s no secret that when it comes to building applications, many developers prefer to use the Apple Macintosh. However, that preference can create multiple DevOps challenges. VMware is moving to reduce the friction with the release of VMware Fusion Pro 10, a desktop virtualization offering for Apple Macs that now supports REST application programming interfaces (APIs) to more easily programmatically invoke virtual machines on Macs.

In addition, VMware has significantly improved support for graphical processors units (GPUs) and 3D graphics within VMware Fusion Pro 10 that are now more commonly being invoked by developers.

At the same time, VMware is also updating VMware Workstation 14 and Workstation Player, which adds support for Windows 10 and Windows Server 2016 along with updated Linux support for Fedora 26 and Ubuntu 17.04 guest operating systems to the desktop virtualization software. Workstation Player is a subset of VMware Workstation 14 that IT organizations can employ to limit the size of a virtual machine running on a local workstation.

Other new capabilities include support for Microsoft’s Virtualization Based Security (VBS) features of Windows 10 Enterprise and Windows Server 2016 guests, enhanced virtual networking and new support and controls for working with vSphere and the vCenter Server Appliance software.

Michael Roy, product line marketing manager for VMware Fusion, says more organizations are making GPUs available to developers as part of an effort to build more robust applications. Support for GPUs make it easier for developers to build compute-intensive applications for multiple operating system environments within the context of a DevOps environment managed by an IT organization, he notes.

Other new capabilities include a revamped user interface and support for the Apple Touch Bar to simplify moving between native applications and virtual machine environments, including applications hosted on the VMware PhotonOS distribution of Linux optimized for hosting container applications.

In this release of Fusion Pro, VMware is also adding support for Microsoft’s VBS features—which include Credential Guard for Windows 10 virtual machines, unified extensible firmware interface (UEFI) Secure Boot and a virtual Trusted Platform Module—in the same way it supports them on VMware Workstation 14.

Finally, VMware is providing access to enhanced networking controls to enable management of network address translation (NAT) rules, network packet loss and latency simulation and renaming of VM networks.

The updates were announced ahead of VMware’s VMWorld annual conference, being held Aug. 27-31 in Las Vegas.

Developers typically require more access to more IT environments than almost any other employee, so maximizing the number of operating systems that can be deployed on their systems is in the best financial interests of the organization. Even when an organization can afford multiple machines most developers don’t want to have to move files between physical machines. Desktop virtualization software provides a means to not only increase productivity but also reduce the amount of machine overhead associated with developing applications across multiple operating environments.

— Mike Vizard

August 28, 2017
Puppet Makes Development Kit Available to Advance DevOps

Organizations that embrace DevOps often find themselves moving from having a paucity of IT automation tools to an embarrassment of riches in short order. As a result, many of the teams involved in those DevOps processes wind up making different IT automation choices. Before too long, the organization finds itself trying to bridge too many islands of automation.

Puppet is working to bridge the automation gap between developers and IT operations teams with the release of a Puppet Development Kit that makes it easier for anyone to generate code that can be executed using the Puppet IT automation framework.

Michael Olson, senior product marketing manager for Puppet, says all too often developers automate various tasks using their own favorite tool. When it comes time for IT operations teams that have standardized on Puppet to deploy that code, they have to rewrite various routines. Puppet Development Kit makes it possible for developers to now write code that can be shared seamlessly with IT operations teams that have adopted the Puppet IT automation framework.

In some environments, DevOps has literally become too much of a good thing. Olson says the Puppet Development Kit represents an opportunity to truly unify fragmented processes in a way that developers can easily consume. The Puppet Development Kit includes unit test modules from Windows, OS X and Linux workstations to ensure that Puppet code is creating and managing configuration resources as intended.

The existence of the Puppet Development Kit should also make it easier for organizations that have not embraced DevOps to get started. A recent survey conducted by Puppet finds that organizations that have embraced DevOps are delivering new applications and updates at rates that far exceed other organizations. The longer it takes for organizations to embrace DevOps the wider that gap grows. By making it easier to include developers in more structured approach to DevOps, many companies that are getting started late with DevOps now have an opportunity to close that gap.

Developers don’t always have a lot of respect for IT operations. Many of them have been throwing code over the proverbial data center wall because deploying code in a production environment was always someone else’s problem. Tools such as the Puppet Development Kit essentially steal a page from the same playbook Tom Sawyer used in the book by Mark Twain to get the kids in the neighborhood to help him paint a picket fence.

At the other end of the spectrum, however, are developers who are firmly convinced they can program the IT operations team out of existence now that “infrastructure is code.” In reality, most of those developers quickly discover that the ongoing care and feeding of hundreds of applications deployed in a production environment still requires a fair amount of human intervention. In fact, without some form of IT automation framework in place, deploying applications at that level of scale would not be possible. The challenge is striking the right balance between what can be automated and higher-level tasks that still require human expertise.

— Mike Vizard

August 22, 2017
CollabNet Merges with VersionOne for ALM Supremacy

Thanks to the rise of DevOps, there has been a lot of focus on various forms of application lifecycle management (ALM) spanning multiple application development methodologies. But as IT organizations become more comfortable with multiple methodologies, it turns out they are not replacing one with another as much as simply trying to figure out where to employ one over another. That, in turn, is driving some consolidation across the ALM category.

CollabNet this week announced it is merging with VersionOne, a provider of an ALM framework optimized for agile application development. While CollabNet has made some forays into the space, historically its primary focus has been on waterfall methodologies commonly used across larger enterprise IT organizations. CollabNet CEO Flint Brenton says the goal now is to combine the best of the ALM tools provided by both companies to enable IT organizations to leverage agile and waterfall methodologies as they best see fit. In fact, he says the biggest ALM issue facing organizations today is the need to balance developing applications faster and the inherent risks associated with potentially moving too quickly.

Robert Holler, formerly the CEO of VersionOne, will assume the post of chief strategy officer for the combined company. He says there’s very little overlap between the two company’s product portfolios, and the one thing that will distinguish CollabNet going forward is an open approach to ALM that enables organizations to employ whatever tools they prefer. Instead of forcing organizations to rip and replace those tools, Holler says ALM frameworks need to foster collaboration rather than trying to impose a closed ALM environment. That environment also needs to be deployable on-premises or in the cloud, depending on the requirements of the customer.

Holler notes that once any segment of the IT industry reaches a certain level of maturity, consolidation is all but inevitable. The opportunity CollabNet has is providing an ALM environment that makes it possible to automatically package and deploy applications that includes strong version-control mechanisms. Those capabilities will become even more critically important as IT organizations expand usage of microservices based on, for example Kubernetes, alongside traditional virtual machine deployments across the enterprise. Accomplishing that goal will require a common management layer capable of spanning both environments, he says.

The combined companies have more than 20,000 customers. That number may not collectively represent the lion’s share of the ALM market, but it does make CollabNet a serious contender against a host of rivals that are both a lot older and larger. CollabNet’s challenge now is convincing customers of competitive organizations to switch to an ALM platform that CollabNet contends is a whole lot more flexible.

In the meantime, it’ll be interesting to wait and see if the combination of CollabNet and VersionOne will drive any additional acquisitions at a time when IT organizations are becoming a lot less religious about what type of application development methodology gets employed by who and when.

— Mike Vizard

August 10, 2017
Sonatype Report Spotlights Software Supply Chain Issues

Most application developers today don’t write much raw code. Rather, applications developed today are created mostly by combing various modules and widgets to create a custom application. But currently there is little oversight being applied to the provenance of application components, especially when it comes to open-source software.

The third annual State of the Software Supply Chain Report published by Sonatype, a provider of tools for managing software supply chains, finds that the amount of time it takes for open-source projects to address vulnerabilities is considerable. The report states that only 15.8 percent of open-source software projects actively fix vulnerabilities, resulting in an average mean time to remediation of 233 days. In effect, that gives cybercriminals the better part of a year to exploit a known vulnerability. Of course, that’s usually longer simply because most IT organizations are not especially efficient when it comes to regularly updating all the components that make up an application once it’s been deployed in production.

Based on an analysis of more than 17,000 applications, the Sonatype report estimates downloads of Java components grew 68 percent (52 billion) year over year in 2016, while JavaScript downloads (59 billion) grew 262 percent.

Matt Howard, chief marketing officer for Sonatype, says managing all the components that make up an application will only become more challenging as developers increasingly start to leverage Docker containers. The Sonatype report estimates there will be 12 billion downloads of Docker containers in 2017, a 100 percent increase over 2016.

The good news is that more awareness of DevSecOps issues appears to be having an impact. The report finds the percent of Java components downloaded from the Sonatype Central Repository that contained known security vulnerabilities fell to 5.5 percent (1 in 18), down from 6.1 percent the year prior. Of course, that improvement needs to weigh against the 68 percent increase in downloads, which would suggest that the total number of components with known vulnerabilities finding their way into production applications has never been higher.

By way of contrast, the report claims that applications built by teams employing automated governance tools reduced the percentage of defective components in their application by 63 percent. The report also says organizations that are actively managing the quality of open-source components flowing into production applications are realizing a 28 percent improvement in developer productivity, a 30 percent reduction in overall development costs and a 48 percent increase in application quality.

The report makes clear that DevOps processes are not being extended far enough to the proverbial left. Instead of focusing on only on the application release cycle organizations need to apply structured processes to the components that are being aggregated together during the application development process. In fact, Howard notes that application development is moving away from being an art to more of a manufacturing process. As that shift continues to occur, it’s only a matter of time before a much greater emphasis gets placed on quality control all the way down that manufacturing line to not only limit liability but also generally improve the overall application experience.

— Mike Vizard

August 8, 2017
Loom Systems Extends Reach of AI into DevOps Realm

One of the things that tends to hold back adoption of DevOps are all the low-level events that need to be correlated by an engineer to generate anything that approaches actionable intelligence (AI). To eliminate that requirement, Loom Systems created a Loom AIops platform to automate many of the tasks that normally would need to be performed by a DevOps engineer.

Loom Systems has upgraded the Loom AIops platform by adding Distributed Entity Tracing, which makes it easier to analyze events across a range of interdependent applications and associated middleware. In addition, Loom has created Correlations View, which simplifies identifying how alerts being generated are related to one another.

Finally, the latest version of AIops adds support for the OpenStack cloud management framework, in addition to sporting a revamped set of dashboards that are easier to navigate.

Dror Mann, vice president of product for Loom Systems, says one of the reasons that DevOps is not widely implemented is that today it requires IT staffs to accumulate and correlate massive amounts of low-level event data to create anything approaching actionable intelligence. Loom AIops is designed to automate the low-level tasks relating to DevOps to enable the IT staff to focus more of their time and effort on fixing issues and reducing the time required to release and update applications.

Loom AIops, says Mann, accomplishes that goal by first streaming all the application log data into its platform, followed by all the log data generated by any dependencies that application might have. Loom AIops then applies machine learning algorithms in real time to identify the root cause of an issue. Armed with that information, it becomes a lot easier to reduce the mean time to resolution (MTR), which then provides the IT organization with the time needed to address more complex management challenges, he says.

Mann notes the single biggest challenge IT organizations encounter with DevOps is that it requires them to change their modus operandi. By automating many of the lower-level DevOps tasks, there’s more time available for IT organizations to think through the cultural changes required to successfully implemented integrated DevOps processes.

Most IT professionals will view the application of AI to IT operations with a combination of skepticism and trepidation. It will be difficult initially for many IT professionals to trust the analysis being generated by AI software. Once they do begin to trust that analysis, there naturally will be some concerns over to what degree an AI program can eliminate the need for a DevOps engineer altogether. But, Mann notes, there are many complex tasks that never get addressed because most DevOps engineers spend an inordinate amount of time and effort trying to correlate the right data.

At this juncture, usage of AI within the context of DevOps is now all but inevitable. The challenge now is not figuring out the line between where humans add higher orders of value compared to machines that are generally going to be limited to identifying patterns. The advantage a machine has it that once it learns something, it doesn’t forget it, take days off or switch jobs. However, being able to identify a pattern is not nearly the same as being able to apply reasoning to what the sum total of those patterns really means for the business.

— Mike Vizard

August 4, 2017
IBM: DevOps Challenges Require Managed Services
The DevOps complexity associated with determining which classes of application workloads are best-suited to run on certain platform types will push an increasing number of IT organizations to embrace managed services. That’s the premise put forth by IBM, which provides such a service.

Steve Robinson, general manager for IBM Cloud Platform Services, says it’s becoming less likely that internal IT operations teams will have the tools and expertise required to analyze and optimally deploy workloads. In fact, the number of platform choices have already exceeded the ability of most internal IT organizations to master. They include:
- Traditional virtual machines: The most widely deployed form a virtual machine software comes from VMware. But there’s already a lot of interest starting to emerge in so-called “micro VMs” that promise to reduce the memory footprint of VMs. However, rather than replacing traditional VMs, it’s probable micro VMs would be deployed alongside traditional VMs.
- Containers: Most containers today are deployed on top of virtual machines because of a lack of tools as well as security concerns. But as IT organizations become more adept at managing containers, many more of those containers will wind up running on bare-metal servers that will be able to attain much higher utilization rates by eliminating the need to deploy traditional virtual machines.
- Serverless Computing: Based on event-driven architecture, these platforms can be invoked natively as functions or simply be used to host containers. In either case, the need to provision servers, storage and networking resources is eliminated because IT infrastructure resources are made available on demand in real time.
- Graphical Processing Units (GPUs): As an alternative to Intel Xeon class processors, GPUs now are taking on many workloads involving machine-learning algorithms and big data because of their ability to process many of those workloads in parallel.
- Field Programmable Gate Arrays (FPGAs): Based on custom integrated circuits, FPGAs are being more widely employed within a variety of cloud services.
IT organizations today have the option to deploy any of these technologies on-premises or invoke them in the cloud. The issue most are contending with is a lack of advanced IT skills. In the absence of those skills, many IT organizations are in peril of falling behind their rivals. To overcome that issue, Robinson says it’s easier for many organizations to rely on third-party providers to manage their infrastructure while they concentrate their limited resources on application development.

Any increased reliance on managed services to gain the benefits of a more sophisticated approach to DevOps will vary widely by organization. Crafting DevOps process that span third-party organizations takes time and effort. There’s no way to analyze a workload and then automatically deploy it on the optimal platform without any human intervention. But as IT organizations continue to struggle with mastering DevOps processes, many may conclude it makes the most sense to rely on the expertise of a third-party service provider to manage a hybrid approach to DevOps. Regardless, the pressure to show results by increasing the amount of high-quality code being generated across the enterprise is mounting.

— Mike Vizard
August 4, 2017
NodeSource Gets to the Source of the Node.js Issue

There’s no shortage of variants of Node.js in the world, but NodeSource is making a DevOps-related case for why IT organizations should standardize on its runtime implementation.

NodeSource released version 2.3 of N|Solid, which adds support for event loop delay notifications that pinpoint precisely where an issue that needs to be addressed has occurred with an application, based on the Node.js variant of JavaScript.

Pravin Halady, product manager for NodeSource, says N|Solid uniquely provides a runtime implementation of Node.js that provides that capability. Coupled with tools to monitor applications based on N|Solid, additional visualization tools and configurable webhooks-based integration to create alerts that can be shared via collaboration platforms such as Slack, N|Solid is the only Node.js runtime specifically optimized for enterprise IT environments, Halady says.

He notes one of the thornier issues organizations that embrace Node.js face is that any long-term synchronous activity blocking the event loop can prevent other incoming requests from reaching the server. Before too long, the application hangs or simply crashes. By providing insights into when an event loop delay is occurring in real time, Halady says IT organizations don’t have to waste time trying to reproduce the problem. The result is a much faster time to resolution of Node.js application issues.

DevOps teams can find themselves dealing with those types of issues at a much higher level of frequency once developers employ Node.js. As adoption of the most widely employed variant of JavaScript increases, IT organizations are managing more applications than ever. In fact, a recent survey published by the Node.js Foundation finds that three out four respondents plan to increase their usage of Node.js over the next 12 months. The primary reasons cited for that additional usage are increased developer productivity (68 percent); improved developer satisfaction (65 percent); reduced development costs (58 percent); and increased application performance (50 percent). Nearly half the respondents (47 percent) working in enterprise IT organizations say they been using Node.js for more than three years. Over half the developers (58 percent), in contrast, report having more than 10 years of experience working with Node.js.

IT organizations generally have very little visibility into what’s occurring inside Node.js applications, which makes it difficult to apply DevOps processes. In fact, most DevOps processes in place today are wrapped around applications written in more mature languages such as Java. Developers employing JavaScript tend to not be as well-versed in the role programmers ideally play within the context of an integrated set of DevOps processes.

In the absence of those processes, having the tools needed to debug an application in real time becomes more important to an IT operations teams that needs to inform a developer about the root cause of a specific issue. Otherwise, the developer ends up running one test after another in the hopes of replicating the problem. Unfortunately, the probability of accomplishing that goal inside most organizations is often relatively low. Because of that issue alone, an event loop delay notification made in real time is likely to pay for itself in a matter of hours.

— Mike Vizard

August 1, 2017