devopsy.kinsta.cloud

Category: Application Performance Management/Monitoring

  • Why Legacy Application Development is a Challenge

    Why Legacy Application Development is a Challenge

    Enterprises seeking to modernize their legacy mainframe applications often cite the key reason as not being able to evolve quickly enough to support the changing needs of the business. This is also a driving force behind the increasing trend toward moving these legacy applications onto other platforms, either by attempting to rewrite or re-platform them.

    If this was a simple process however, the transition would have happened years ago. So, why is it so difficult to modernize mainframe applications? It is surely not the programming language. COBOL and other traditional languages are simply programming languages. They may not be considered “cool,” but neither are many of the other scripting or development languages, or configuration syntax programmers deal with day in and day out.

    So, what actually makes legacy application modernization challenging? After countless conversations with several outsourcers, system integrators and mainframe customers, four major issues came up time and time again.

    Development Tooling–Low Productivity on Mainframe

    The traditional environment for a programmer, working with COBOL or PL/1 on the mainframe, has few of the productivity features today’s programmers expect: smart cross-references, on-the-fly syntax checking, autocomplete for variables etc. Attempts have been made to bring the experience closer to that of a modern JAVA programmer, but there is no escaping the fact a mainframe exists in the background with all its quirks. This divergence in programmer productivity unfairly tends to stick to the programming languages on the mainframe. With the result that language choice gets conflated with developer tooling in decisions to make mainframe development more agile.

    Open-Source Availability–Limited Access to Innovation

    Looking for solutions to business requirements among the many popular open-source projects is one of the fundamental reasons why development processes in general have become more agile. Clearly, if less code is needed to support any requirement, the faster that requirement will be satisfied. Very few open-source projects have been implemented and tested on mainframes. Perhaps even more troubling is the difficulty of integrating a Linux-oriented solution with legacy applications in a natural way.

    Pipeline Integration–Separate Pipeline and Processes for Mainframe Development

    In a similar vein to developer tooling, modern development pipelines do not fit naturally with the way in which mainframe applications move from development into production. Build processes, testing, source-code management and promotion are integrated and streamlined within a modern development pipeline. Specifically, modern tools enable developers to promote locally-tested changes and have those changes flow through an automated testing pipeline, with steadily increased levels of integration with existing applications and data. The pipeline is facilitated by the tight integration of tools such as GIT, Eclipse, Jenkins, Containers, etc. to leverage the best practices of CI/CD. All working in harmony with an agile development organization to get application updates to production faster. The mainframe is not as efficient, mainly due to nascent support for many of these technologies, alongside the costs of running many of the automated processes repeatedly.

    Testing–Limited Autonomy and Automation

    Testing, as part of a modern development pipeline is a highly automated and scalable process. The ability to launch containers with a single click is a fundamental reason why development is faster today than in the mainframe era.

    Development expectations for testing extend to being able to launch a complete testing environment on the developer workstation, without recourse to systems administrators, for the recurring small unit tests intertwined with the development itself. Due to the physical hardware dependency of mainframe development, this testing autonomy can be highly impractical for legacy applications.

    Furthermore, large scale-tests, which can be regularly, automatically and cost-effectively scheduled in modern, container-based cloud environments, require extensive budgeting and planning for legacy applications.

    Consequently, all testing of legacy application developments is extended by the need to involve mainframe systems administrators, and schedule expensive mainframe resources.

    Solving the Issues

    To address the issues above, a solution is required which breaks the software lock-in that ties a legacy application to the mainframe. It is possible to rewrite or re-platform existing applications, although this can be time-consuming, costly and risky. A more beneficial approach is to rehost the mainframe applications by migrating them to the cloud, without recompilation or conversion of data types. This method replaces the proprietary, mainframe hardware and software dependencies with a set of Linux libraries, requiring less time and fewer resources to execute.

    Linux libraries can be created to include features that recreate the behaviors of the operating system, databases, security and language runtime that previously tied the application to the mainframe hardware. In addition, the incompatibility of the mainframe instruction-set with x86 hardware can be managed via a virtual machine.

    If the mainframe is represented as a set of libraries in the same way as any other more conventional application dependency, then the entire problem of legacy application development is reduced. Any workstation or commodity server can fully virtualize the mainframe environment, in several instances in parallel–if needed–thanks to containers. And with total elasticity when on the cloud.

    Migrating to the cloud makes it possible for a developer to spin up a full test environment for legacy applications on their workstation, in the same way as they would for a Java or C# program. The entire development cycle for legacy applications can follow the same pipeline as for other applications. The same agile models, and CI/CD DevOps policies with strong emphasis on shift-left testing can be employed. The same automated scale testing is possible. The same exploitation of open-source projects can be considered.

    Once rehosted, the word “legacy” can be dropped from the description of these applications. They simply become applications written in COBOL or PL/1. Everything else about their ongoing enhancement is identical to those applications written in modern languages.

    The highlighted issues that businesses seeking agility are facing, are only issues that exist on the traditional mainframe. Migrating legacy applications to a software defined mainframe, running on an x86 architecture provides a modern dynamic environment in which to develop, innovate, integrate and test applications.

    — Dale Vecchio

  • New Relic Extends Observability Reach and Scope

    New Relic Extends Observability Reach and Scope

    At its FUTURESTACK 2019 event in New York today, New Relic announced it has extended the New Relic One Observability Platform to add support for its new Logs, Traces, Metrics and AI module offerings.

    Ken Gavranovic, general manager and senior vice president of products for New Relic, said his company is committed to extending the breadth of a programmable platform to address DevOps requirements flexibly, rather than forcing IT teams to stitch together multiple tools to monitor their applications.

    As part of the effort, New Relic also is making it possible for customers to build applications using the data New Relic stores in its cloud platform using a framework based on React and GraphQL. The company is also making available a series of open source monitoring applications that organizations can customize in less than an hour using templates it provides.

    New Relic also announced the availability of New Relic Logs, a log management service intended to make it easier for operations teams to correlate log data across application and infrastructure sources. The module will enable teams to run ad hoc searches that return queries in milliseconds without having to switch to another tool.

    At the same time, New Relic announced the availability of New Relic Traces and New Relic Metrics, to help DevOps teams more easily identify dependencies across the IT environment. The goal is to make it possible for teams to understand all the upstream and downstream events occurring across a distributed IT environment using a single platform, said Gavranovic.

    For example, DevOps teams will be able to monitor a serverless computing framework from within New Relic in addition to turning on a feature flagging tool that has been integrated via application programming interfaces (APIs).

    Finally, New Relic AI, available in beta, will bring artificial intelligence (AI) to the platform in the form of machine learning algorithms and predictive analytics to correlate, aggregate and prioritize incident data automatically across multiple tools to reduce alert fatigue and improve mean time to resolution. New Relic AI is already integrated with PagerDuty, Slack and other tools to enable organizations to slipstream this capability into their existing workflows, said Gavranovic.

    By taking that approach, organizations can embrace New Relic as an open platform by organizations regardless of their current level of IT maturity, he added.

    New Relic is clearly betting a single integrated tool for monitoring the entire IT environment will be preferable to integrating a wide range of disparate tools. Gavranovic believes that approach will have even more appeal as organizations embrace microservices to build and deploy applications. Given all the dependencies that exist between all those microservices, monitoring tools will need to reach across applications and infrastructure to enable DevOps teams to discover and monitor all those dependencies in near real-time.

    Naturally, New Relic is not the only provider of monitoring tools with similar ambitions. It is, however, a lot further along in terms of turning monitoring tools into an actual platform.

    — Mike Vizard

  • Splunk Adds Omnition to DevOps Portfolio

    Splunk Adds Omnition to DevOps Portfolio

    Splunk followed up on its pending acquisition of SignalFx, a provider of an application performance management (APM), by also acquiring Omnition, a startup building a distributed tracing tool accessed as a software-as-a-service (SaaS) application.

    Rick Fitz, senior vice president and general manager for the IT Markets Group at Splunk, said Omnition consists of a small team of developers who have been especially active in maintaining OpenCensus, an open source set of libraries for various languages that make it easier to collect application metrics and distributed traces and then transfer the data to a backend of choice in real-time; and OpenTelemetry, a project led by the Cloud Native Computing Foundation (CNCF) that merges OpenCensus and OpenTracing to standardize the instrumentation and telemetry data collection.

    Distributed tracing is emerging as a requirement for APM platforms because the IT environment is becoming both more complex and ephemeral. Serverless computing frameworks are especially challenging because the only way to know an event occurred involving, for example, cloud bursting is to employ distributed tracing. The challenge is, distributed tracing requires organizations to capture, store and analyze massive amounts of data to pinpoint the root cause of any application performance issue, noted Fitz. As such, distributed tracing tends to be employed only by the most elite DevOps practitioners, as implementing distributed tracing is nontrivial, he added. On the upside, distributed tracing reduces the need to deploy a large amount of agent software across the enterprise.

    Fitz said Omnition will provide a set of capabilities that over time will be added to the SignalFx platform. Long-term, Splunk hopes to reduce the costs associated with distributed tracing to the point where that capability becomes democratized, he added, noting that as organizations embrace microservices to build cloud-native applications that will run in hybrid computing environments, increased reliance on distributed tracing is now all but inevitable.

    In general, Fitz said Splunk is working toward combining the best elements of DevOps with traditional IT service management (ITSM). Most organizations will be employing both approaches for years to come simply because some applications don’t need to be updated regularly, so it doesn’t make any sense to apply DevOps processes to those applications, he noted. At the same time, as DevOps processes are embraced in enterprise IT environments, appreciation is growing for IT operational challenges that stem from multiple applications running at scale which need to share a limited amount of IT infrastructure resources.

    With the acquisitions of SignalFx and now Omnition, Splunk is betting that it will be able to combine IT operational analytics based on machine data generated by logs with an APM platform to correlate application performance against infrastructure events. It’s not clear how long it will be before Splunk can fully realize that vision. In the meantime, the rate at which application and infrastructure management platforms will converge will only accelerate as developers are held more accountable for applications long after they are deployed in a production environment.

    — Mike Vizard

  • Splunk to Acquire SignalFx to Gain APM Platform

    Splunk to Acquire SignalFx to Gain APM Platform

    Splunk today announced it will acquire SignalFx as part of an effort to extend its reach beyond IT operational analytics and into the realm of application performance management (APM).

    Valued at approximately $1.05 billion, the deal to acquire an APM platform delivered as a software-as-a-service (SaaS) offering will expand and accelerate Splunk’s overall transition to a recurring revenue model based on cloud services, said Splunk CEO Doug Merritt during a call with industry analysts.

    Splunk is currently on pace to generate over $300 million per year from the cloud version of its IT operational analytics platform. Splunk revenues for the second quarter were $517 million, up 33% year over year, with software generating $350 million of that total. However, as Splunk transitions to more of a subscription-based pricing model, the company has some financial management challenges. GAAP operating loss for the company’s second quarter was $87 million. However, non-GAAP operating income was $47 million, even though operating cash flow was negative $129 million. That cash-flow issue is a direct result of the transition away from a perpetual software license model based on the volume of data consumed by the Splunk platform. Overall, Merritt said Splunk is firing on all cylinders and is still on track to have 20,000 customers by the end of its fiscal year.

    Merritt said once the deal is approved, sometime in the second half of fiscal 2020, SignalFx will provide a complementary set of services for monitoring legacy monolithic applications as well as cloud-native applications based on containers, Kubernetes and serverless computing frameworks. There are very few customers at this point who have not adopted cloud-native technologies, but for now they remain a small percentage of the overall number of application workloads being managed, he added.

    The goal is to provide a suite of services that will enable IT teams to investigate, analyze, monitor and act on data in real-time, said Merritt.

    Splunk is already widely employed by internal IT operations teams, many of whom already rely on a complementary APM platform. Splunk clearly expects organizations will prefer to acquire an IT operational analytics platform and an APM service from the same vendor. However, competition between APM platform providers is already fierce. That said, Splunk appears to be counting on increased demand for real-time application monitoring to differentiate SignalFx from rivals.

    It’s not clear to what degree Splunk might converge the various services now being provided separately by both companies. It’s also not clear how or whether this deal will accelerate acquisitions of the remaining independent APM platform providers.

    In the meantime, Merritt said SignalFx and Splunk’s earlier acquisition of VictorOps, provider of incident management platform based on best DevOps practices, will expand the amount of visibility Splunk has with application development teams.

    Ultimately, acquisitions of smaller, DevOps-oriented companies by vendors that today are still more closely identified with legacy ITIL-based IT processes, such as Splunk, are only like to increase. What will be interesting to watch is whether the tools and services being acquired by those companies wind up being the DevOps tail that wags the proverbial ITIL dog.

    — Mike Vizard

  • Catchpoint Expands Network Monitoring Service Reach

    Catchpoint Expands Network Monitoring Service Reach

    Catchpoint has expanded the number of locations worldwide from which it is gathering telemetry data for its networking monitoring service to more than 825 sites.

    Drit Suljoti, chief product officer for Catchpoint, said as applications continue to become more distributed, the worlds of DevOps and network operations (NetOps) converge. DevOps teams need more visibility into how network latency issues anywhere in the world are impacting the digital experiences of their end users, said Suljoti.

    The Catchpoint monitoring service now extends to 200 cities worldwide, including 140 nodes in mainland China. In addition to monitoring critical backbone locations, Catchpoint has placed nodes in 133 public cloud locations and is now offering a free and seamless migration for companies looking to adopt a next-generation digital experience management (DEM) platform in place of Gomez and Keynote services from Dynatrace that recently were retired, said Suljoti.

    In much the same way DevOps teams rely on a variety of tools to monitor applications, they also need access to a service that continuously monitors the networks on which those applications depend, Suljoti said. That requirement has become more critical as the number of latency-sensitive applications such as those based on microservices increases, noted Suljoti. As organizations depend on those applications to drive a variety of digital business processes, visibility into how the network is impacting the application experience of end users can easily degrade, he added.

    Most organizations already have tools to monitor network performance within their own data center. However, creating the monitoring framework required to identify network performance issues across a global wide area network (WAN) would be cost-prohibitive for most organizations. When it comes to deploying applications that span multiple WANs, most DevOps teams are essentially blind.

    It’s still not clear within many organizations who is responsible for monitoring WAN services. Historically, NetOps teams have held this responsibility. However, Suljoti said that as DevOps processes continue to expand and mature, there is a more collaborative approach emerging that incorporates network monitoring tools within testing frameworks that organizations create within the context of a continuous integration/continuous deployment (CI/CD) process.

    In the meantime, pressure on existing WAN networks will only increase. In the absence of 5G services, the number of applications competing for access to bandwidth increases. Many organizations have no idea whether network service providers are really meeting the commitments they agreed to as part of the service level agreement (SLA) provided. As more application performance issues get traced back to the underlying network, frustration with the quality of network services is only going to mount. Before they can call network service providers to account, DevOps teams will need access to analytics that clearly show service level commitments are not being met.

    Of course, that may only elicit an apology from the network service provider. At the very least, however, DevOps teams will be able to make a much more informed decision concerning how they want to route application traffic going forward.

    — Mike Vizard

  • Autonomous Security in Containers

    Autonomous Security in Containers

    With the advent of DevOps, the development world has quickly moved to agile development practices and containerized applications. At Forum Systems, we have responded to this trend by putting our API security software, Forum Sentry, into virtual form factors such as Amazon Machine Image, Azure Image, VMware Image, Linux, Windows and Docker.

    Why do we feel it’s necessary to containerize API security in this way, and what benefits does it deliver over traditional software or even hardware-based API security?

    What Is API Security?

    Before we explain why we containerized our API security software, it is first necessary to define exactly what we mean by API security. As a result of the major IT trends of the last decade–in particular cloud, mobile and IoT–more people and applications are connecting to IT assets than ever before.

    What’s more, the majority of these interactions are from untrusted entities outside the organization’s network perimeter (and as you will read below, across container boundaries). Securing your organization’s systems, data and business-critical processes is harder than ever.

    Almost every interaction relies on an API to communicate to an application or system somewhere in the world. The simplicity of APIs makes it easy for developers to connect their projects to other systems to enhance their functionality, with data being easily shared between a myriad of external partners, cloud providers, virtualized data centers and on-premise applications.

    As a result, APIs have become the primary channel for business transactions and the traditional boundaries of data exchanges have become blurred. APIs provide many benefits but also present risks, as wherever there is innovation, there is also the dark side of threat and attack, and always someone who will aim to exploit weaknesses.

    In many environments, APIs and their underlying technologies are designed primarily to share data; they are not designed to thwart threat and attack. This is where API security gateways are necessary to validate both the data and the identity of users, systems and devices interacting with the API.

    API security goes far beyond just access control, it requires specialized technology to perform dynamic data security to inspect and ensure the specific and unique characteristics of each API communication are correct. The API security policies are designed to prevent intrusion, data leakage and other forms of data loss as a seamless and transparent part of the technology architecture.

    API security gateways differ greatly from API gateways; the key word “security” immediately distinguishes this technology from simple API gateways, which serve only to provide proxy and simplified access control points. API security gateways are purpose-built to protect against API threats, and the dynamic data security capabilities they offer provide centralized protection of bi-directional API communications. API security gateway technology provides essential risk mitigation at each layer of the API architecture, and as rising adoption of containers and micro-service architectures arise, it provide zero-trust concepts within and across the container boundaries to protect the assets and technologies therein.

    Why Containerize API Security?

    Container technology such as Docker has become a popular means to deploy API micro-services, a collection of loosely coupled services which are fine-grained and lightweight. The move toward virtual and cloud was initially driven by fully virtualized images with their own operating system, but the adoption of lightweight services and on-demand environments has led to widespread adoption of container technologies run on a shared operating system. These container architectures provide flexibility and ease of deployment, but come with the same set of API risks.

    The concept of containers also brings into focus the trust model that needs to be considered for communication among technology components within and across containers. Containers become a new API boundary layer and thus represent the same type of risk paradigms, where a rogue container application can gain access or otherwise wreak havoc if the proper security controls are not put into place (commonly known as an insider attack).

    By deploying API security directly into a container, organizations can automate their API security into their existing workflows and provide zero-trust security capability that ensures real-time enforcement and protection of container-based APIs. This means API security becomes autonomous and baked into the architecture, rather than tacked on at a later stage.

    As mentioned earlier, dynamic data security is essential wherever information is traversing. The traditional cyber umbrella approach no longer applies in the interconnected API world. It’s like using an umbrella in a water park, it may stop the water from the top, but not from the sides. By having automated API security at the container and virtual layers, the ability to visualize, connect and secure information becomes an integral (and necessary) aspect of the container security strategy and a fundamental baseline for secure API enablement and trusted data exchanges.

    API security gateways deployed as container nodes allow DevOps teams to harness the power of virtualization, cloud computing and containers while at the same time protecting the organization against API threats from both the inside and the outside. With API security baked into the container, developers are free to focus their time on building the best functionality in their apps.

    — Jason Macy

  • Speaking the Language of Applications

    Speaking the Language of Applications

    I arrived early to a meeting, and the pleasant security guard at AT&T suggested I take a look at their innovation museum while I wait. Among motion picture sound technology and life-size photos of switchboard operators on roller-skates stood a transistor prototype. The transistor was invented in Bell labs in 1947, and it changed the world. Steve Wozniak considered it the greatest innovation of all time in his 2014 article.

    But almost nobody needs to think about transistors anymore. The transistor had to become transparent and abstracted, so the next generations can build great things in higher level. The transistor made a huge influence on our lives as an enabler, but if people had to keep talking about transistors it wouldn’t be possible.

    Abstracting technologies is key to advancing to a higher level, making it possible to solve real business problems.

    The language of a client-focused business is applications. If we want to make a business execute faster these days, we need to make the development and release of its applications execute faster. But one thing stands in our way: the infrastructure. Applications reside on infrastructure–they are installed on servers, bundled in containers, on cloud instances, in networks, using storage.

    If we want to speak the language of applications, we need to abstract them from the infrastructure, and make the infrastructure transparent.

    The concept of abstracting applications is what stands at the basis of every new approach to application development and release around us.

    • Containers are all about abstracting services from specific infrastructure and making them mobile.
    • Serverless takes it to the extreme.
    • Public cloud makes it easier not to worry about owning physical infrastructure.

    Technology screams, “Think applications, soon you will not have to worry about infrastructure!”

    And yet, most of us mix applications and infrastructure in an unhealthy way when we try to automate them. It is unhealthy because mixing applications and infrastructure prevents reuse of automation assets and makes it very difficult to connect infrastructure usage to a business context.

    Understanding how mixing applications and infrastructure is manifested requires looking back into the evolution of infrastructure and application automation.

    The Evolution of Infrastructure: Manual Deployment and the Inception of Its Automation

    In the beginning there was manual. We got a server; it was sitting on a shelf. We got the application artifacts, maybe on a CD or a USB stick. We installed the application on the server. Our application was ready for use. When we needed to upgrade, we went back to the shelf and installed the update on our server. It was very hard to make this operation faster.

    This is where automation of applications and infrastructure started.

     

    Behind any automation tool or standard stands a model. This model reflects how its creators originally perceived the problem they were trying to solve. It consists of the entities they defined as first-class citizens, the operations they wanted to apply on these entities, their attributes.

    “All models are wrong, some are useful,” said George E. P. Box in 1976. This is true for automation models too. Automation models are usually nice and clean when we start off. But as the problems we try to solve transform, our models need to grow and adapt. Until at some point they are not useful anymore and we need to replace them with next generation modelling approaches.

    The Configuration Management Model

    Configuration management tools such as Puppet and Chef were originally built to manage application deployment and configuration on static infrastructure. This new approach to Ops optimized manually updating applications on servers. Better network connectivity made it possible for this model to thrive, with the ability to configure applications on servers remotely.

    But within a decade we witnessed the fast rise of immutable infrastructure. Immutable infrastructure is an approach to deploying applications where the infrastructure components are replaced instead of updated. This approach somewhat pulled the rug from under the configuration management model. With each update or change, the application must be re-deployed. This forced adding dynamic infrastructure provisioning to the configuration management model. Combining infrastructure automation and immutable infrastructure concepts into the configuration management model is a difficult task, since this model was never designed to accommodate something like that.

    The Infrastructure-as-Code Model

    Pure Infrastructure-as-Code tools were built to provision infrastructure dynamically. Terraform is an example for an open source tool that does this well. This model is created with infrastructure in mind.

    Whether we look into Terraform, AWS, CloudFormation or ARM templates, we will see the infrastructure is the starting point (defining network connectivity, VM instances and services) and the applications are an implicit addition. They are artifacts that happen to be deployed on and are tightly coupled to the infrastructure. We may look at a component that says “App A” but it actually stands for “specific infrastructure with app A installed on it.”

    This seems natural, because it’s the same way it worked in the good old physical-infrastructure world: We fetch a box, we install something on it. We focus on the infrastructure, because this is our starting point. The application is the artifacts we put on the infrastructure and scripts we run after we provision it. Presumably, the application cannot live without something to be on.

    Infrastructure-as-Code tools evolved the way open tools do–their base model was extended by the community because there was a need to automate applications deployment and not just infrastructure. But it is becoming difficult to extend this model to reflect a world where applications are built to move between clustered servers, and the starting point is no longer the infrastructure, but rather the application’s requirements.

    Mixing Infrastructure and Applications

    Trying to fit infrastructure provisioning and application deployment in a single model that was originally designed for just one of them creates issues. Infrastructure is lower level than applications. Applications reside on the infrastructure. The same app can reside on different infrastructure configurations. This critical relationship is completely flattened if you mix them.

    Moreover, from a business perspective it is important to make applications explicit. It’s our product. Our website. Our mobile app. The applications need to be the center of the attention and discussion. This kind of modelling puts applications in the shade of infrastructure.

    Four Great Reasons to Model Applications as a First-Class Citizen Right Now

    The models we use today are broken, and it is time to evolve. No matter if your applications reside in containers, or on cloud instances, or include native cloud services or a mix of them. Applications should be a first-class citizen, and not some artifact installed as an afterthought on the infrastructure we started from.

    The notion that applications should be abstracted from infrastructure doesn’t eliminate all the challenges of infrastructure provisioning and application deployment, but it creates a foundation for better and more scalable automation.

    1. Achieve Application Re-Use

    Modelling applications as a first-class citizen is the only way to make applications a re-usable element throughout the development and release pipeline.

    If there’s one thing I learned in my automation journey, it is that if automation doesn’t get frequently and consistently reused, you don’t get value for your money.

     2. The Path to Multi-Cloud

    Abstracting application is also the best approach for moving application workloads between cloud providers. If everything starts from infrastructure, we need to start from scratch when we move. If our applications are abstracted and defined separately—this definition stays the same when we move, only the infrastructure changes.

    3. The Path to Breaking Silos

    The vision of Dev, Sec, and Ops all dealing with the same language (code) is wonderful. But bringing everyone to the same code pool is not always easy. Applications and infrastructure are not the same thing, even though we can code both. Mixing them makes them unnecessarily hard to build and troubleshoot. It forces us to deal with fuzzy boundaries when it comes to responsibility for this code. Basing automation on a model that organizes the world in a way the different stakeholders can understand it, can help overcome this barrier.

    4. Connect Infrastructure to Business Needs

    Infrastructure-as-Code is a huge breakthrough, but the bigger story is about making application development and release faster. This doesn’t undermine the importance of infrastructure; infrastructure is the foundation of everything. But if we want to solve business problems, we need to talk the language of the business, and that’s the language of applications. Our automation should be built in a way that reflects this.

    — Maya Ber Lerner

  • Debunking Myths Around RASP

    Debunking Myths Around RASP

    Runtime application self-protection (RASP) has taken a fair bit of scrutiny over the last few years. Like many security technologies that pioneer new ways of tackling old problems, people inherently don’t like change. Several companies entered the space early and early adopters helped mature various RASP solutions on the market and the technology has advanced rapidly.

    When applications began to be the favored target of bad guys, tons of companies started to reposition their perimeter security products closer to the application. Web application firewalls (WAFs) are similar to IPS for the network, so easy mind shift for most security practitioners. The problem is they require you to know your adversary. That’s like asking a security guard to keep out a single troublemaker, while letting in their associated gang of thugs.

    As we saw recently with WebLogic, exploiting applications is a shell game. When one vulnerability is exposed and patched, another one, which shares the same modus operandi, comes out. When reports came out comparing the most recent WebLogic vulnerability to vulnerabilities found last year in CVE-2018-2628, CVE-2018-2893 and CVE-2017-10271, Oracle posted a blog explaining last year’s vulnerabilities had been patched. The problem was the underlying code mechanism that led to the above-mentioned CVEs (insecure deserialization) had not been fixed.

    Deserialization is a perfect illustration of why RASP is the Chuck Norris of application security. Instead of applying a patch (which in the case of Oracle’s WebLogic, blacklisting the exploit method via a patch), RASP looks at the totality of the code before it executes and says, “If it looks like a duck and quacks like a duck, apply a roundhouse kick to the entire enumeration.”

    It’s important for enterprises to protect their customers from insecure deserialization with RASP products. Further, it’s critical to consider learning and debunking some of the common misconceptions in the overall perception of runtime security:

    Performance Impact

    Evaluate an agent-based solution, which often makes app owners concerned it’ll impact the performance of their applications. When evaluating RASP providers, especially agent-based solutions, ask for agent size, performance benchmarks and certainly conduct your own performance testing during evaluation.

    Writing Secure Code

    With the DevOps movement, companies have ramped investments in training and programs around secure development practices. And while security is shifting left with this movement, programming still has a little bit of art infused, especially when developing applications that have been around for a long time and have seen variations in development resources. As is the case with WebLogic, application developers should create apps with security in mind, but they don’t have a crystal ball and exploits evolve just as applications do.

    My Developers Will Become Lazy

    Somewhere in these early days of RASP, the notion came about that RASP means you no longer need to remediate vulnerabilities in your application source code. Notably, the root cause of all vulnerabilities should be fixed as soon as possible. However, RASP can also function as an insurance policy allowing DevOps teams to better plan fixes and prevent derailing other projects focused on the business.

    Why Do We Need RASP If We Have a WAF?

    The WAF has become ubiquitous in most organizations and it is certainly valuable for organizations looking to take a defense-in-depth strategy. WAFs have become a significant strain on resources as they produce unmanageable amounts of false-positives that require constant tuning. In a recent survey by Ponemon, it takes most organizations 45 hours a week to address alerts and 16 hours per week creating or updating rule sets.

    Hopefully, this article illustrates more color to today’s RASP. We’ve seen a lot of advancements in the early days of this technology and hopefully that will mean a lot less headlines.

    — John Adams

  • Datadog Fetches Bevy of Additional DevOps Services

    Datadog Fetches Bevy of Additional DevOps Services

    At its Dash 2019 conference today, Datadog launched a series of DevOps initiatives that significantly expand the use cases for the agent software that drives the company’s monitoring services.

    Additions to the Datadog portfolio include Network Performance Monitoring and Real User Monitoring (RUM) services as well as general availability of a previously announced service for monitoring serverless computing frameworks.

    In addition, Datadog announced it is adding Tracing without Limits to its application performance monitoring (APM) and distributed tracing service that allows DevOps teams to search all traces without live sampling and store the traces matching bad user experiences for the long-term.

    Finally, Datadog is adding support for log rehydration and the ability to generate custom metrics from ingested log data residing within its log management service.

    Ilan Rabinovitch, vice president of product and community for Datadog, said the company continues to invest in expanding the core use cases for which its agent software can be applied. Over time, that approach will enable DevOps teams to rationalize the types of agent software they need to deploy to monitor applications, cybersecurity, infrastructure, networking and user experience, he said.

    Rabinovitch noted that interest in extending the reach and scope of Datadog services into the realm of networking is especially high because DevOps teams today have little visibility into how the various software-defined network (SDN) services provided by cloud service providers are impacting their applications.

    DevOps teams are also now being asked to present IT metrics in a way that reflects actual end user experiences. The RUM service being provided by Datadog achieves that goal by making it easier to track front-end latency metrics such as time to interactive and time to first byte, in addition to providing a geographic location, the operating system and browser types being employed by the end user.

    Rabinovitch noted that as organizations become more dependent on digital business processes, they are moving to instrument every aspect of delivering IT services. For example, APM in the past might have been limited to a handful of mission-critical applications running on-premises. Now, as more digital business applications become distributed across both cloud and on-premises IT environments, the need for visibility across interdependent applications and microservices has become more acute, he said, adding most organizations would prefer to consume monitoring as a service versus having to deploy and maintain that software themselves.

    It’s unclear how being able to apply a common base of agent software might accelerate the convergence of job functions within an IT organization. But it does make it easier for cross-functional IT teams to collaborate and troubleshoot IT environments so all members of a team can use a common management platform to identify issues. In many cases, the amount of time saved by eliminating the need to convene a “war room” meeting whenever a new issue is discovered might in and of itself be priceless.

    — Mike Vizard

  • The Linux Foundation Breathes New Life into Osquery

    The Linux Foundation Breathes New Life into Osquery

    Facebook’s useful but neglected DevOps tool Osquery has gotten a new lease on life, thanks to The Linux Foundation.  

    Anyone who has been tasked with monitoring the security of server instances in a data center or cloud knows how laborious and time-consuming it can be. Osquery, a project started by Facebook, aims to lessen this burden by reframing how developers engage with their infrastructures. DevOps professionals can use Osquery to expose an operating system as a high-performance relational database, making it possible to use SQL commands to access data about a system, just as they would for a database. 

    Osquery works on Mac, Linux and Windows systems and is provided as an open source download via GitHub. Although Osquery was developed by Facebook to monitor and safeguard the security of its own platform, the social media giant quickly realized the utility of the platform would extend to other enterprises that depend upon insight into the low-level behavior of operating systems. 

    Facebook publicly released the project’s code and documentation in 2014, in response to feedback from a small number of other companies that tested Osquery. The public availability of Osquery resulted in a number of major companies including Airbnb, Dropbox, Netflix, Etsy and Uber to bring Osquery into their software development environments. Nevertheless, users have voiced frustrations with Facebook’s handling of Osquery, and have accused the internet giant of neglecting the project.

    To address those complaints, The Linux Foundation has taken over the mantel of Osquery and recently announced the formation of a new foundation to support the Osquery community. 

    The new Osquery Foundation brings together engineers and developers from Dactiv, Facebook, Kolide, Trail of Bits, Uptycs and many others committed to supporting the project with technical contributions and long-term stewardship. A Technical Advisory Board will be created to handle the shift in management from Facebook to the Osquery Foundation and will oversee the priorities initiated by the foundation’s members.

    This restructuring of management promises to extend the usability of the project as a number of developers contribute to the codebase.

    “Osquery has drastically simplified the process of operating system monitoring, which has unlocked new methods for securing infrastructure, detecting anomalies and more. We look forward to working with the Osquery community to develop the foundation in a manner that will continue to foster the growth and adoption of Osquery while supporting the community’s diverse needs,” said Jim Zemlin, executive director at the Linux Foundation.

    Developers working to maintain the security of their systems believe it would be wise to incorporate Osquery into their workflow. “Although Osquery will never do everything that a security team needs, it gets about 80% of what you need for endpoint insight in one package,” said Doug Wilson, director of security for Uptycs.

    Although the platform requires some configuration, developers familiar with SQL will feel accustomed to using Osquery to retrieve information about their systems. Furthermore, Osquery’s cross-platform support and customizability make it especially attractive to businesses with newer types of infrastructures and ones that work at scale. 

    The move away from Facebook toward a more community-friendly foundation is sure to breathe new life into the project, and as development on Osquery expands, it is likely to become a more valuable resource for security teams moving forward.

    This article was co-authored by Tyler Ohlhorst, research analyst and freelance copy editor and copywriter at Magnum Consulting.

    — Frank Ohlhorst