Tag: application management

  • Cloud Visibility Remains a Top Challenge for Many Organizations

    Cloud Visibility Remains a Top Challenge for Many Organizations

    A number of enterprises have a love/hate relationship with cloud computing. Because of its low upfront costs, the expense for cloud is lower, especially in the early years. But historically there has been another cost: the challenge of obtaining good visibility into applications, users and network traffic. I thought advancements in cloud management and security tools in recent years had largely overcome these concerns. Turns out, I was wrong—or, perhaps, market perception hasn’t caught up to reality.

    A recent survey from Dimensional Research, which is based on responses from 338 IT professionals, found that fewer than 20% of IT professionals believe they have complete, timely access to data packets in public clouds. In private clouds, the situation is somewhat better, with 55% reporting adequate visibility. Unsurprisingly, on-premises data centers had the most positive response when it came to data visibility.

    The survey authors said the results show companies have low visibility into their public cloud environments, and the tools and data supplied by cloud providers are insufficient. They identified some of the challenges such lack of visibility can cause, including the inability to track or diagnose application performance issues and monitor and deliver against service-level agreements, as well as delays in detecting and resolving security vulnerabilities and exploits.

    Some of the findings regarding cloud visibility from the survey:

    • 87% of respondents expressed fears that a lack of cloud visibility is obscuring security threats to their organization.
    • 95% of respondents said visibility problems had led them to experience an application or network performance issue.
    • 38% cited insufficient visibility as a key factor in application outages and 31% in network outages.

    Interestingly, 99% of respondents identified a direct link between comprehensive network visibility and business value. This shows IT performance contributes directly to the business bottom line. Here are the benefits respondents cited regarding visibility:

    • Monitors and ensures application performance (60%).
    • Enables better threat identification (59%).
    • Helps to identify security indicators of compromise (57%).
    • Helps identify performance degradation.
    • Enables threat protection tools to identify malicious traffic by source.
    • Enables security monitoring tools to detect indicators of compromise.
    • Enables the monitoring of traffic at every link of their network.

    A full 86% of respondents also stated visibility was important for network and application performance monitoring, and 93% stated it was valuable for security. Cloud consumers said predicting application performance was a key challenge.

    Still, the lack of perceived visibility isn’t holding back public cloud adoption, with 84% of organizations having increased their cloud usage in the past year. While 3% of those surveyed had a moderate decrease in cloud usage and 13% remained unchanged, 63% experienced a moderate increase and 21% experienced a significant increase.

    As organizations engage in digital transformation efforts and continue to turn to public clouds, organizations can’t afford to lose visibility. This is crucial for all of the business benefits of cloud traffic visibility the survey respondents cited, such as attaining superior application performance, meeting SLAs and shrinking the time it takes to detect and remedy security relation issues.

    But here’s an unpopular opinion: There’s not much of an excuse for having poor visibility into cloud workloads anymore. There once was—that’s for sure—but today there are many tools made available by cloud service providers and third-party security vendors.

    Today, it’s a matter of will and finding the right toolsets, putting them into place and making sure that security and operations teams have the software controls and the training they need to use them, so the organization can attain the level of visibility necessary.

    — George V. Hulme

  • Solving the ‘Automagical’ Apps Automation Problem

    Solving the ‘Automagical’ Apps Automation Problem

    The DevOps Revolution started with a handful of  simple principles. A key one was that systems should be managed by code, not by humans, to eliminate complexity and ensure better security, compliance and scalability. To manage compute infrastructure at any scale, the process should be automated with logic captured in code. And, ideally, automation also would apply as you move up to the stack to middleware and applications layers.

    Unfortunately, it’s not that simple. In fact, automating applications turns out to be way more complex than automating server deployments. For this reason, we need a new approach to managing applications and infrastructure. Managing servers and lower-level environments could be simpler, but existing bottom-up methods work fine. Managing applications, however, must come from the top down. That is, applications should come with everything they need to be successful: configuration, packaging, discovery, scaling requirements and more. And that just isn’t how we have historically built software. Applications are built on layers and layers of dependencies that connect all the way down to the system level. So how do we break the mold and make an application “self contained?

    Thousands of Apps with Millions of Dependency Permutations

    By automating the Linux and Windows server ecosystem and a small number of  derivative operating systems we can cover roughly 95 percent of operating systems, and the series of functions that are common across them. The complexity arises when we try to lay that matrix upon the array of applications that exist in the real world of deployment and production. In smaller companies, you usually have at least a dozen applications and often more. In larger companies, you are dealing with hundreds or thousands of applications.

    This melange spans bespoke apps running in legacy languages, proprietary third-party software and more modern open source software. There are literally millions of permutations that must be automated across dependencies, along with security and compliance regimes. In the world of Kubernetes, for example, each container requires its own Sidecar that contains configuration and management information. Kubernetes Sidecars, however, don’t have robust automation capabilities and quickly sprawl out into a mass of human-defined management instructions that can easily conflict or break down under stress.

    The upshot of this? Today we can write a small amount of code that can effectively manage hundreds of thousands of servers for Facebook. This is true to basically infinite scale. But to automate management of Facebook’s software has injected orders of magnitude more complexity.

    Fear of Application Complexity Killing Cloud Migrations

    This dichotomy has led infrastructure automation to go sideways; organizations have adopted  inauthentic automation coupled with departures from the real ideals of DevOps and agile methodologies. Many companies just aren’t realizing the true benefits of infrastructure automation because they are doing something that is needlessly complicated and may have characteristics of automation but in fact is even more complicated. Thus, they have lost the value of the word and merely exchanged one complexity for another.

    Similarly, many application development teams are doing what they call “ad hoc” infrastructure automation as DevOps teams avert their eyes and pray. Translation? A roll-your-own mishmash of tools to manage containers or VMs, applications and all the dependencies from the top to the bottom of the application runtime environment. What this really translates into is a human executing an ever-changing set of narrow paths against an individual server—a recipe for chaos. This chaos is why the majority of our customers believe infrastructure automation is a critical step to achieve agility but a smaller subset have fully embraced it.

    It’s also a major reason why many companies that would love to migrate to the cloud have not yet done so. Managing the repackaging and movement—the “lift-and-shift”—of all their applications is a daunting prospect that they rightly believe will be far more difficult than simply switching from bare-metal or VMs to instances in AWS or Azure or Google Compute Engine.

    Now, we don’t need to throw the baby out with the bathwater. The automation of operating systems and VMs and containers works just fine and is actually a tremendous improvement on past efforts. In fact, we could easily operate there with a smaller set of features and capabilities than we have to date. But applications must be managed from the top-down to accommodate the inherent complexity of managing so many more moving parts.

    Enter The New Paradigm: Top-Down Application Management

    Top-down application management works in a fundamentally different way, Rather than rigid configurations and choices about what an application should do, modern application management tools for DevOps teams should ask what an application needs to succeed and do its job well across the entire life cycle, from development to production to retirement. These tools should automatically discover changes in dependencies of an application stack, from front end to middleware to database and load-balancing and security.

    If there are changes in these dependencies or other critical factors, then those tools need to automatically rebuild and test the applications, and have them ready to deploy either as a canary or directly into production if the changes are minor. For example, if a new SSL patch is published and a DevOps team updates its underlying infrastructure, then the application management toolchain should recognize which applications need to make which changes, and how that might affect both upstream and downstream dependencies. Ultimately, the toolchain should quietly and seamlessly fix all affected applications with the patch but avoid service interruption.

    Once the success criteria of the applications are defined by a human DevOps person, then the application configuration and management tools’ job is ensure that criteria is always met. Collectively, this is a new type of packaging that captures the best of all possible worlds, merging configuration and packaging into intelligent application management that can scale and flex as needed.

    By capturing the dependencies of configuration and the strictures of traditional packaging, top-down application management methodologies and tools allow a CIO to move an aged C or C++ back-office application into a cloud-native environment just as easily as a passel of Node.js microservices backstopping a mobile application, and have them both run with the same level of continuity and flexibility. Which is really what everyone wants—a safer, easier and better way to run applications without having to worry about the underlying infrastructure.

    Conclusion

    In a sense, then, the DevOps Revolution is entering a new phase. As microservices continue to slice and dice the old functionality of monolithic apps into numerous smaller apps, the challenge of managing this app proliferation will only grow—and become more complicated.

    The way we thought about DevOps tools in the days when cloud computing was relatively young was understandably naïve and not yet informed by the vast application cornucopia we would face in subsequent years magnified by the reality that legacy applications decades old would continue to run in corporate data centers until their code keels over. Now we know and we can design an automation methodology and practices that map to our modern realities. We can do this by giving infrastructure the simple rules and tools it needs, while giving apps more leeway to be themselves and succeed regardless of what is happening at lower levels of the stack.

    — Corey Scobie

  • Perforce Acquires Perfecto Mobile to Drive DevOps Consolidation

    Perforce Acquires Perfecto Mobile to Drive DevOps Consolidation

    Perforce this week announced it plans to acquire Perfecto Mobile, a provider of automated application development tools, in a move that signals a potential wave of consolidation across the DevOps sector. Perfecto Mobile is the fifth acquisition Perforce has made in the last two years, and company CEO Mark Ties said there will be more to come.

    Schedule to close later this year, the acquisition is estimated to be about $200 million.

    The primary driver of this wave of consolidation is access to inexpensive capital coupled with mainstream adoption of DevOps processes that is requiring many startup companies to scale beyond their means, Ties said.

    As a provider of a cloud-based set of tools for automating the testing of web and mobile applications using a variety of open source frameworks, Perfecto Mobile will complement a portfolio of application testing tools based mainly on the technologies Perforce gained when it acquired PRKA earlier this year. Perfecto Mobile currently has more than 3,000 customers.

    Application testing, along with application development and deployment, increasingly is shifting to the left as developers assume more responsibility for the overall application environment. Instead of relying mainly on a dedicated team to test applications after they have been developed, companies now are continually testing modules as they are developed. Sometimes that’s done by the developers themselves, while other organizations prefer to maintain testing teams made up of individuals who did not write the code being tested. Proponents of the latter approach contend that approach leads to higher-quality code because the individuals testing the code are not as invested in how the code was written. Of course, in an ideal world, an organization would be able to embrace both approaches.

    Regardless of the approach taken, the one thing that is clear is that the rate of at which applications are being developed requires more reliance on automated testing than ever. Not only are there more applications being developed, but increasingly each one is made up of microservices. There often are multiple versions of those microservices that are subject to frequent change. To make matters even more challenging, the number of dependencies between microservices that make up an application are increasing exponentially. That means the days when organizations could rely mainly on manual testing are coming to an end.

    There may come a day when machine learning algorithms are able to play a bigger role in automating application testing and DevOps. But for now, that day is still far off. In the meantime, most organizations are still incorporating multiple approaches to automated testing into their application development processes. The issue is no longer so much about whether to automate testing as much as it is to what degree.

    In the meantime, the more acquisitions that occur across the DevOps landscape, the greater the validation of DevOps becomes as vendors scramble to extend portfolios that, in many cases, are still optimized for legacy approaches to building and deploying applications that increasingly are falling by the wayside.

    — Mike Vizard

  • 9 Essential Ingredients for Modern Enterprise Applications

    9 Essential Ingredients for Modern Enterprise Applications

    Digital transformation. It’s the latest in a series of buzz phrases intended to describe how enterprises are embarking on top-to-bottom modernization of their IT environments. It’s easy to see why it’s a hot topic. Emerging technologies including cloud, containers, microservices and distributed applications hold the promise of greater flexibility, lower cost and faster release cycles. The benefits to companies that make this journey can be enormous.

    Unfortunately, making this journey can take years with today’s manual processes and antiquated tools. Not to mention that most IT budgets are at least 80 percent focused on keeping the lights on. Application modernization projects can be extremely complicated since legacy, monolithic enterprise applications are intrinsically dependent on underlying infrastructure and data stores. Cloning complex application environments, including critical production data, for upgrades or testing often can take days or weeks, slowing down cycles and contributing to lower quality releases. Even new applications are too often designed with infrastructure dependencies, trapping organizations onto specific cloud, server or container vendors. This vendor lock-in reduces choice and leaves IT organizations open to unexpected price hikes down the road.

    The truth is that the way we have designed, built, deployed and managed enterprise applications needs to change. Business-critical software is too often built in monolithic blocks that are nearly impossible to move to new infrastructure or storage options without lots of effort and time. The applications are hard to upgrade since updating even one small portion means rebuilding and retesting the entire thing. Managing the various versions of the discrete application modules and the resulting testing permutations can be quite challenging. It’s not unusual to hear from CIOs that they feel trapped in a world where they can only reasonably expect to update critical enterprise applications just twice a year at most.

    Business needs to move faster. There’s no reason why release cycles, even for enterprise applications, couldn’t be every few days instead of months. Application quality should actually be better in this world. Continuous delivery models such as DevOps should be an integral part of the process. And security should not be an afterthought; it should be built-in. To accomplish this, the future of enterprise applications must look very different from today.

    What are the essential ingredients for modern enterprise application platforms and tools in this new world? Here are a few suggestions.

    Complete Portability, Without Code Change

    Enterprise applications must be abstracted and separated from any dependency on underlying infrastructure, including sources of data. By abstracting the functions of an application into components that can run anywhere, it’s possible to then move the application to different infrastructure combinations, without changing a single line of code. It’s only through complete portability between cloud environments, container tools and servers that IT organizations will break vendor lock-in and gain the flexibility required to move their applications to vendors that offer the best combination of price, performance, reliability and features.

    On-Demand

    All enterprise applications should be on-demand, meaning the provisioning of new instances should be hands-free and fully automated. This implies tight integration with orchestration tools that most companies are already using to provision infrastructure. The key difference here is that the provisioning of storage options also needs to be fully automated. This is made possible by the fact that modern applications won’t have any dependencies on the specific data stores they rely on. This on-demand nature is not restricted to technical teams but includes business users that can easily perform user acceptance testing by simply provisioning full-fledged application environments.

    Self-Healing

    Modern enterprise applications should be self-healing, which means applications not only should have measurable data points to define their health but can then react based on these metrics. They must be able to scale up (or down) based on performance characteristics up to a certain cost threshold, restart on a healthy node in case of infrastructure bottlenecks and cache or deploy closer to the user based on latent use performance.

    Composable (DevOps-friendly)

    All enterprise applications should consist of a number of building blocks: APIs, web servers with content, databases seeded with data, security policies, the operating system, network configurations, deployment configurations and application binaries. By breaking apart monolithic enterprise applications into smaller components that can then act as independent services, IT organizations gain the flexibility to upgrade and automatically deploy individual components through standard DevOps processes and orchestration tools. The services can run anywhere desired, giving flexibility of choice in infrastructure or sources of data. And developers can use a point-and-click UI to pull pre-defined components together to build larger functions of an application.

    Model, Snapshot and Clone

    Once an application is separated into its essential components, a catalog can be maintained that makes it easy to create new versions of the application. Developers, test engineers and DevOps teams can then pick and choose whatever combinations of components they need to create or clone complex application environments for testing or deployment. The state of the application environments can be snapshotted, cloned and replicated. This cloning process should be efficient and quick. This dramatically speeds up development and testing processes, which to-date have been largely manual.

    Scalable

    Scaling application and infrastructure environments automatically, on-demand, and in near real-time is one of the essential capabilities that makes modern infrastructure so valuable. Any solution that is used to develop, test, deploy and manage enterprise applications should offer these same capabilities, giving the ultimate in cost flexibility, performance and scalability. As containers are moving quickly from the lab into production environments, look for a container-based solution that complements this strategy.

    Security is Included

    Application security should not be tacked on after deployment. This slows down continuous delivery processes like DevOps and creates friction between development/ops and the security team. Instead, security should be considered an essential component of the overall application environment and treated the same as any other component and baked into the application from the start.

    Push-Button Upgrades

    Upgrading enterprise applications should be made much easier by taking a point-and-click, push-button approach through a modern UI and integration with continuous deployment tools. Once an application is blueprinted into its various components, using a modern UI to choose versions and assemble the final application becomes entirely possible, saving time and reducing manual errors.

    Single-Pane-of-Glass Application Management

    By taking an end-to-end view of the enterprise application development, testing, deployment and management process, it’s now possible to create a “single pane of glass” console that gives development and deployment teams a powerful way to manage what’s been deployed to various infrastructures in the organization. With this approach, it’s also now possible to collect invaluable utilization and cost metrics which can help an IT organization continuously improve their portfolio.

    Enterprise applications of the future are going to look very different from today’s monolithic behemoths. By leveraging new technologies and processes, IT organizations will be able to dramatically speed up their modernization efforts and release cycles, improving competitive position. The inherent cost savings of this approach will be transformational as well, as IT budgets shift toward net-new innovation, not just keeping the lights on. The future looks bright indeed.

    About the Author / Rahul Ravulur

    Rahul Ravulur is the cofounder and CEO of appOrbit, which was founded in 2014. Most recently, he was the Director of Product Management at VMware managing the Availability suite of products. Before that, Rahul managed the vCenter Server and Client set of products with a special focus on scalability and extensibility to chart the evolution of vCenter to a broad virtualization management platform. Prior to VMware, Rahul held engineering leadership roles at BMC Software and Marimba.

  • F5 Networks’s State of Application Delivery, 2017

    F5 Networks’s State of Application Delivery, 2017

    For the last three years, F5 Networks has surveyed its users and presented the “State of Application Delivery” report. This year’s report is already a couple of weeks old, and I’ve had the opportunity to read through it. There is good information in there for “NetOps” people—those who are focused on the network but want DevOps-style management. The report discusses the trends and what its customers—which lean to the larger, more established organizations—are doing.

    DISCLOSURE: I’m a former F5 employee, and overall, I loved the company. Its upper management has managed to navigate bad times and thrive in good times, and the direction of the company was generally excellent while I was there. My reasons for leaving had nothing to do with top-level management or product. In short, I’m a fan of F5. Also, my wife is still an employee of the company and—worse than that—had a hand in creating both the survey and the resulting document. And Lori sometimes also writes for devopsy.kinsta.cloud. Thus, any criticisms I offer here I will have to be willing to defend on social media, at editorial meetings and at the dinner table.

    With all that said, since readers can download the report and form their own opinions, I’m comfortable writing about this report. And I’ll even risk doing battle over the steak fries for my loyal readers.

    F5 asked nearly 2,200 users of its gear from a variety of industries a series of questions regarding how they use application services—those things such as load balancing, web application firewalls or DDoS protection that exist for an application but aren’t part of it. Included also were DevOps questions and automation questions outside the realm of DevOps.

    While much of the report is of general interest for any IT practitioner, some of it is specifically of interest to DevOps practitioners. So, with the exception of a few sweeping statements, I’ll restrict myself to the DevOps-relevant portions of the report.

    Insider side note: Lori has mentioned F5 has a metric ton of data it was unable to squeeze into this report, and she would be talking about off-and-on for a while at her F5 blog. If this topic interests you, it might be worthwhile to follow her.

    The first thing to visit is the overall list of takeaways. The report offers four “Key Findings” and, interestingly enough, three of them are relevant to DevOps. When you dig into the fourth (security-related) key finding, you discover that it, too, is at least tangentially related to DevOps.

    While I don’t want to give away the entire report, I will mention the three items directly related to DevOps:

    1. Digital Economy drives application services adoption.
    2. Cloud is growing and skills are needed.
    3. Cost savings are the primary driver for DevOps in their survey base.

    An increase in application services that can be automated kind of strongly implies DevOps, while cloud and DevOps go hand in hand, and the last point explicitly is DevOps.

    The report is packed with information about what F5’s customers are doing, what’s important to them moving into 2017 and what they plan to implement soon. Overall, it is a great read, and might bolster your appeals for executive backing or increased funding.

    There are some oddities in the report, though. I’m not a fan of the editing; some sentences seem clunky (using the same word twice in a sentence with different assumed context, for example); and one list of percentages is for some reason organized least to greatest, while all others are organized greatest to least.

    While editing errors are distracting, the biggest issue I had with the report was the inclusion of Python as a “framework” (along with VMware, OpenStack, Puppet, Chef …). The assumption is made in the accompanying text that this indicates the level of homegrown automation, but since there are many ways to implement homegrown automation, this is at best a low point indicator of that. Overall, it feels out of place in the list:

    DevOpsTools
    DevOps tools as surveyed by F5 Networks.

    But honestly, there is enough other information about trends and number of these “frameworks” in use per org packed into this section that I’m willing to view inclusion of a language as a simple oversight that can be explained away because lots of organizations are using python for automation (I have, at more than one organization).

    The report also talks about the average application having double digits of application services deployed for it. This seems high to me, but it is data from a survey of F5’s customers, and if you’ve already paid for this bundle of services, perhaps you are more likely to make use of as many as possible. What this points to, for me, is an automation of the overall application environment. Considering the commentary F5 received in response to skills questions, it makes sense that automating solutions that can easily be automated would be an easy choice for their respondents.

    Note that in the graphic above, the company did not restrict users to “What frameworks are you using for network configuration?” It might be a reasonable assumption to think it did, as F5 is primarily a networking company (don’t tell the execs I said that—they seem to be repositioning as an application services company these days), but I asked, and the company put no limits on responses. That shows uptake of Puppet among its customers to be far smaller than I would have expected, personally, considering the core of its customer base should be Puppet’s sweet spot—traditional enterprises in a variety of industries.

    The weighting of respondents across roles might have some play in how these are ranked, though. There is a preponderance of networking and management types in the response list, with fewer security, dev, architecture and DevOps types. This could skew results, even if the organization list is relatively balanced across industries.

    Overall, take a look. It’s solid information, with some interpretation available and more commentary on verticals and use cases. It’s definitely a document worth having on hand when explaining the value of what you’re doing, or attempting to expand what you’re doing.

    — Don Macvittie