Tag: enterprise applications

  • 7 Benefits of Self-Contained Apps for BPM

    7 Benefits of Self-Contained Apps for BPM

    According to Gartner, “As business needs change, organizations must be able to deliver innovation quickly and adapt applications dynamically—reassembling capabilities from inside and outside the enterprise. To do this, organizations must understand and implement the ‘composable enterprise.’”

    This is an appealing concept for companies looking for a good balance between highly centralized and highly distributed technologies and information systems.

    For enterprise IT, the use of composable infrastructure – built up of independent components that can be swapped out when needed – is already widely applied. Another place to look for the advantages of composability: Business process management  (BPM) automation. Why? Because IT teams, DevOps teams and leadership are facing a number of pressures:

    • Business teams need responsiveness and speed from IT as everyone continues to look for ways to maximize go-to-market.
    • Current business process automation platforms deploy applications that are all dependent on shared resources. The DevOps team must take interoperability and integration among a group of applications using shared resources into consideration for new developments, new deployments and maintenance. This can be complex, time consuming and can require extensive testing before updates can be deployed.
    • Similarly, whenever the process application platform is upgraded to a new version to take advantage of new features, or when maintenance on a single application is needed, all applications are affected and the need to offline and upgrade everything can impose a high cost in downtime.
    • Scaling grouped business process applications can add unnecessary cost as it is applied across the board even if not all of those applications need to be scaled.
    • The approach of building business applications from interoperable elements promises to address some of these key issues.

    Modularity and component-based systems have already been successfully applied for many years in IT and software development. Infrastructures are composed; we no longer buy machines but CPU, memory, disk space, bandwidth and so on. In software development we see composability applied through containerization, an approach that is already widely used and becoming the standard for DevOps teams, as shown for example by the growing use of platforms such as Docker and Kubernetes.

    Self-contained applications is setting the same trend for composable BPM process-based applications. BPM technologies already fit perfectly into the approach of composable architectures. Applications themselves are also already internally made up of composable elements: Processes, sub-processes, connectors, services, reusable pieces of code, business rules and UI fragments, just to name a few.

    Self-contained apps combine the benefits of containerization approaches and technologies with the power of its BPM application platform technologies for application development. The BPM base offers wide extensibility to integrate with enterprise IT, with good auditability, traceability and compliance; deployment of individual applications using containers offers faster delivery, portability and easier management.

    DevOps teams can deploy complex applications more easily, deliver continuously, and thus support innovation and faster time to market. Each self-contained app can be deployed individually, independently and quickly to deliver increased value through improvements as quickly as the business requires.

    DevOps teams, IT leadership and CIOs will be happy to see self-contained apps because:

    • The containerized approach included with self-contained apps is already familiar to CIOs and their technical teams.
    • The less complex, independent development and deployment of self-contained applications means faster application delivery to production overall. As it’s not necessary to take interoperability and integration among a group of applications into account during development, development is faster, testing is easier and the resulting application is easier to maintain.
    • Scaling an independent application is simpler and it can be applied to each application individually.
    • Application maintenance costs are reduced, as smaller and more regular updates are now possible.
    • Because it’s easier to implement multiple projects, development teams’ efficiency and productivity, and therefore ROI, increase.

    Business will be happy to see self-contained apps because:

    • Automation and business application projects are more easily adapted to business changes; the IT team can respond more quickly and with more confidence due to the smaller scope and less interdependence among other applications.
    • Faster innovation and competitive advantage offers a faster response to new business needs.
  • ERP Modernization: How Devs Can Help Companies Innovate 

    ERP Modernization: How Devs Can Help Companies Innovate 

    In this post-pandemic world, large enterprises are looking to streamline and improve business processes within a storm of marketplace constraints. Companies in asset-heavy industries such as oil and gas, energy, retail, or distribution and logistics have especially been working to reconfigure their supply chain processes to better manage inventory and operations. The focus in the market has shifted from supply chain optimization to supply chain resiliency, creating efficiencies that will help them best survive current delays and rising costs. 

    Breaking Barriers and Addressing the Talent Shortage

    The question is not simply whether companies need to modernize their business applications to increase efficiency and lower costs, but how to accomplish a necessary modernization in the midst of barriers like rising costs, supply chain disruptions and a severe shortage of developer talent. Developers and DevOps teams need to find ways to do more with less.

    The reimagining of business processes is an excellent strategy to reduce overhead and allow companies to become nimbler and more efficient, consequently helping them preserve or improve marketshare. Many large enterprises work in environments that use enterprise resource planning (ERP) software; systems that, over the years, have been subject to ongoing upgrades, patches and redesigns by developers and IT managers. This creates technical debt, which makes it difficult for many of these large-scale companies to migrate their business applications to cloud-based systems.

    Many of these ERP-based companies are facing pressure to update to more modern, cloud-based versions of their ERP platforms. But they must run a gauntlet to modernize their legacy applications. In a sense, companies that maintain these complex ERP-based systems find the environments are like “golden handcuffs.” They have become so complicated over time that they restrain IT departments’ innovation efforts, hindering their ability to create supply chain resiliency when it is most needed.

    To make matters more difficult, the current market is facing a global shortage of human resources required to get the job of digital transformation and application modernization done, including skilled ERP developers—especially those skilled in more antiquated languages like ABAP. Incoming developer talent is often trained in more contemporary languages like Java, Steampunk and Python. These graduates have their pick of opportunities and gravitate to companies that already work in these newer programming environments. 

    ERP migrations can be hampered by complex, customized systems developed by high-priced, silo-skilled programmers. Many enterprise leaders have a great desire to break out of these restrictions and modernize legacy applications with greater speed and at lower cost, and they need to do so with minimal downtime and disruption. 

    Leverage Low-Code Solutions to Modernize

    In a 2022 report on application development strategies, industry analyst Deepa Salem interviewed enterprise companies that used sophisticated SAP-based ERP solutions about their approach to software modernization strategies. More than 80% of respondents said they planned to upgrade their business applications in conjunction with a major migration of their ERP software platform to the latest, cloud-based version. 

    This tactic of implementing a new applications strategy in coordination with a major refresh is not surprising and presents an opportunity for developers to help these companies “clean house,” reducing the technical debt of their legacy environments at this well-timed juncture.  

    As an analogy, think about how people typically move to a new residence. Homeowners sift through their accumulated possessions and identify what’s no longer needed. They pack what is still valuable and essential. A low-code application process can help clean up existing ERP environments prior to migration, reducing complexity and long-accumulated technical debt while leveraging the system’s unique business intelligence. The tribal knowledge inherent in any legacy system is critical to retaining market differentiation. 

    With so much at stake, IT departments and developers should give serious consideration to using pre-built, customizable business applications, efficiently and rapidly created through a low-code development process. Deployment-ready solutions can help developers move forward with modernization projects that have been hampered or shelved due to a lack of talent to implement those jobs. They can accomplish more with less.

    With the right approach, ERP-based environments can be reconfigured to quickly establish more agile, streamlined and cost-effective business applications. Developers have a unique opportunity to help enterprises keep what sets them apart from the competition and surrender what no longer serves them. In this climate, this is not a nicety for enterprise customers. It’s about resilience—and even survival.

  • Are ERP-Centric Enterprises Being Left Behind?

    Are ERP-Centric Enterprises Being Left Behind?

    Many large enterprise companies are rife with departments that depend on customized ERP software applications. These systems accommodate vital business tasks ranging from inventory management to ordering capabilities to accounting programs to CRM platforms—all of which were custom-built to fit the enterprise’s specific needs and meet the KPIs of each unique organization. These applications tend to sustain grades, patches and reworks, sometimes resulting in multiple redesigns over the course of decades, and these often are not properly documented. To make matters worse, these ERP software systems also tend to be written in outdated programming languages such as ABAP for SAP and PL/SQL for Oracle, making them a challenge to update.

    For example, 91% of respondents in a recent ASUG survey of 110 SAP companies said they use custom code, and 41% of those applications were created at least six years ago. The majority of these companies said that the existence of these custom code environments has become a hindrance to digital transformation and is holding back innovation.

    These conditions create the bloated and overly complex legacy environments that are now commonplace in the enterprise market. Widespread, customized core business systems have made digital transformation particularly daunting for enterprise companies. They require significant developer resources, excessive time commitment and ample expense to strip down and redesign these custom systems for the cloud. 

    A Lack of Developers—and of Developer Interest

    U.S. Department of Labor statistics from December 2020 indicated that there will be a shortage of 40 million skilled workers worldwide, amplifying these issues further. That figure is predicted to grow to 85.2 million by 2030. What’s more, new developers entering the marketplace gravitate to more contemporary programming languages such as JavaScript and Python, as opposed to wanting to revamp archaic code used in most ERP solutions, such as ABAP.

    For this reason, large enterprise companies in a range of verticals are lagging behind when it comes to upgrading mission-critical software applications to the cloud or to a more modern architecture. These organizations are well aware they stand to gain significant productivity benefits from digitally transforming their systems so that, for example, inventory data can be accessed via a mobile device or shared from anywhere on a cloud-based platform. Yet these large organizations are stalled in their efforts to modernize. Some companies have gone as far as to scrap their current systems and start reprogramming their software applications from scratch—a hugely expensive, lengthy and laborious endeavor that’s also full of risk. 

    Leveraging Low-Code to Simplify Enterprise IT Environments

    One method that has shown progress in unraveling these overwrought environments is to apply automation through low-code integration. Ideally, an effective low-code development solution will automatically convert existing business processes to modern, cloud-native applications, allowing the organization to maintain the business logic it has built into those applications over the years, while drastically reducing the time commitment, costs and potential disruption associated with traditional, long-form integration. In SAP environments, for example, using a low-code rapid application development solution can separate the application layer from the core SAP system, automating a large portion of the migration while preserving a “clean” SAP core.

    With many organizations under pressure to keep budgets under control in what continues to be an uncertain economy, these large-scale companies are looking for ways to modernize their operations on a more controlled schedule. When enterprises use low-code solutions—and furthermore, don’t have to search for developers of these more archaic languages—they’re free to approach digital transformation in a far more incremental and cost-effective way.

    For instance, the 12 million users of SAP’s solutions are being compelled to upgrade to the modernized version of its product, S/4HANA. Yet the American SAP User Group (ASUG) research shows that only 33% of these organizations have taken on the challenge so far despite the looming end-of-life of the current prevalent version. This is because lacking a low-code development solution, companies are being pressured to convert and rebuild all their business processes in one fell swoop. Few organizations have the resources to absorb such a drastic and risk-prone undertaking. 

    A rapid application development program acts as a bridge for companies that are looking to modernize their businesses, allowing them to upgrade their applications incrementally, according to their own schedules. Low-code can be a catalyst for technical debt-ridden enterprises, allowing them to simplify complicated legacy environments and get down to the business of innovating their systems.

  • Engineering Applications for DevOps (Part 3)

    Engineering Applications for DevOps (Part 3)

    This blog series explains how to engineer applications for DevOps.

    Topics covered in this blog series:

    • Factors used to decide whether an application is a good candidate for DevOps.
    • Practices to engineer designs for DevOps.
    • DevOps applied to enterprise apps and software services.
    • DevOps applied to COTS systems.
    • DevOps applied to manufactured (embedded) systems software.
    • DevOps applied to software services.
    • Five levels of application maturity.

    This blog covers the topic of DevOps applied to enterprise apps and software services; part three in the series. You can read part one here and part two here.

    DevOps Applied to Enterprise Apps

    An enterprise application assists an organization in solving business problems. Enterprise applications are typically designed to interface or integrate with other enterprise applications used within the organization and to be deployed across a variety of networks (internet, intranet and corporate networks) while meeting strict requirements for security and administration management.

    Proprietary enterprise applications are usually designed and deployed in-house by specialized IT development teams within the organization. However, an enterprise may outsource some or all the development of the application and bring it back in-house for deployment.

    The size and complexity of large enterprise applications, especially older monolithic-architected applications that are not designed in modules, are a challenge when it comes to implementing DevOps. Given the large scale and distributed nature of the people, process and technology components of enterprise applications, it is recommended that enterprises engage very experienced DevOps consultants to determine strategic requirements, roadmaps and plans for DevOps that fully consider all three dimensions (people, processes and technologies) and nine pillars of DevOps before embarking on a major DevOps transformation journey.

    Despite the complexity, with the right strategy and plan, DevOps can deliver enormous ROI for enterprise applications. Amazon and Adobe are just two of many examples of large corporations that have benefited enormously from applying DevOps across the enterprise.

    DevOps Applied to Software Services

    I often hear people ask, “Does DevOps apply to software services as well as it does products?” These are usually people who have only worked in product-oriented organizations, such as companies that manufacture things that have embedded software.

    Ironically, I often hear people ask the opposite question, “Does DevOps apply to products with embedded software as well as it does to services?” These are usually people who have only worked organizations in which software is consumed as a service over an intranet or internet for a price or for free. Examples of these organizations are IT departments (even the IT departments of manufacturers!), government agencies and enterprises such as banks, insurance companies, hospitals and financial trading companies.

    You already know the answer to these two questions if you read the earlier parts of this blog series. The answer to both is clearly ‘Yes!’ Cloud services are doubly interesting to DevOps because not only are cloud services used by DevOps as a preferred infrastructure, as described in the next chapter, but cloud services also use DevOps in the process of creating and deploying software for the implementation of cloud components.

    While we are on the topic of DevOps and services, I want to help clarify some confusion regarding whether “newer” DevOps renders “older” IT service management (ITSM) obsolete or less important. This confusion exists because ITSM has often been regarded as a gatekeeper run by Ops to prevent Dev from getting out of control. While some organizations may have, in fact, implemented ITSM that way, this is not the purpose of ITSM.

    “DevOps tends to focus on the software delivery life cycle and on building and delivering software,” said Jayne Groll, president of the ITSM Academy. For its part, ITSM tends to have a broader focus; it is really looking at all aspects of customer engagement, from fulfilling standard services to ongoing support and operations activities. So, ITSM is “almost like the left and right ends of DevOps.”

    Using ITSM With DevOps

    Here are some recommended engineering practices for using ITSM with DevOps:

    • Training to help DevOps professionals gain an understanding of ITSM, and vice versa, will help both teams find points of integration—not just in technology, but in process, as well.
    • Find ways to automate service change approvals implementation of known, good changes to allow for agility, iteration and innovation. At the same time, hold back high-risk, high-impact changes that might need more than just an individual contributor to sign off on before they make it in front of a customer.
    • Self-service requests, automation flows and IT orchestration are now the front door to IT service management actions. These can and should be happening in the background, automagically created, maintained and measured by our DevOps tools.
    • Embrace Agile service management to take Agile values and apply Scrum-like development practices to process design and improvement. Aim to make more frequent incremental changes rather than design and implement a total end-to-end process in the traditional way.

    What This Means

    This blog is part three of a series to explain how to engineer applications to be most suitable for DevOps. This blog series explains how to engineer applications for DevOps including:

    • Factors to decide whether an application is a good candidate for DevOps.
    • Practices to engineer designs for DevOps.
    • DevOps applied to enterprise apps and software services.
    • DevOps applied to COTS systems.
    • DevOps applied to manufactured (embedded) systems software.
    • Five levels of application maturity.

    This blog covered how DevOps can be applied to enterprise apps and software services. You can read part one here and part two here.

    For more information regarding how to engineer applications for DevOps refer to my book Engineering DevOps.

  • Serverless Computing Brings New Security Risks

    Serverless Computing Brings New Security Risks

    Attracted by lower costs and less operational overhead, serverless computing is an unmistakable undercurrent in the world of DevOps. Developers are drawn to the innovation because it requires no architecture to manage while offering continuous scaling for anything from a few requests per day to hundreds of thousands of requests per second.

    More than 50% of the developers using serverless functions today started using the computing model in the last three years. By some estimates, serverless computing is the fastest-growing segment in the cloud services market.

    While businesses are drawn by the opportunity to extend and integrate cloud applications at greater scale, there’s a dark underbelly to this DevOps phenomenon: security. In fact, organizations cite security concerns as the leading reason why they haven’t yet adopted serverless computing. For good reason!

    Serverless functions are introducing a new breed of threats – misconfigurations, code-level risks, injections and weaknesses – many of which are the underlying risk factors for a software supply chain attack. With adoption of this technology accelerating, organizations need security that delivers more than just visibility into these dynamic environments. They also need the ability to detect and block code-level vulnerabilities and misconfigurations that occur during development and through to production. Most importantly, this needs to happen seamlessly and with little to no impact on DevOps.

    Dynamic Environments Create Complex Security Risks

    The nature of applications has evolved, creating a new paradigm of security risk. In serverless environments, code sprawl makes it harder to identify vulnerabilities and patch them before they become business-level risks.

    In a monolithic application environment, an identified vulnerability could be efficiently patched by the developer through upgrading a single package, pushing a deployment and then testing and deploying the app.

    In a serverless computing model, patching and resolving vulnerabilities becomes a lot more complicated, given that there may be several hundred (or more) environments across different providers and regions. So, instead of having to patch a single application, developers will need to patch, deploy and test hundreds of individual functions that rely on that library.

    With 39% of developers saying they’re now fully responsible for security, it means they’ll carry the burden of having to patch an identified vulnerability – often taking them away from development work.

    With Serverless, You Need to Rethink Your Approach to Security

    Given the distributed nature of serverless functions – essentially, the reason for its flexibility and scalability – many existing security tools will provide little to no visibility, nor the control capabilities, for these computing environments. Many of the security attacks that will occur in serverless functions will be a result of misconfigurations and mistakes that happen outside the purview of the security team and due to legacy solutions which don’t translate to serverless architectures. Further, because abstracted workloads create blind spots, attackers will have more room to maneuver undetected.

    Serverless functions will even render some traditional DevSecOps tools less useful. Scanning tools must monitor hundreds of individual repositories instead of a single monolithic repository, while application performance monitoring (APM) tools lack security proficiency and cannot protect from the OWASP Serverless Top 10 risks.

    The challenge for security teams will be implementing solutions in a way that does not modify workloads or require changing business logic — both of which slow development. When applying security to serverless functions, it’ll be critical that the solution can do the following:

    • Go beyond monitoring and actually detect and block vulnerabilities without elaborate or manual steps involved. The scale of serverless functions will make it nearly impossible for security analysts to manage and keep up with every alert they receive. Instead, they need a tool that delivers actual security and can help them block malicious activity without stopping development altogether.
    • Enable a positive security model that provides protection against malicious changes, like zero-day exploits, within the function. This approach gives security teams more control by automatically detecting and blocking anything that isn’t explicitly allowed in a given function.
    • Provide forensic detail from log-level information so security analysts can fully understand the context of every attack with virtually no impact on latency. Further, with the risk of supply chain attacks soaring, security teams need the ability to identify and map third-party dependencies used during runtime.

    For many organizations, serverless architecture is a very different and unique computing environment – unlike anything they’ve experienced or had to protect before now. That reality means that organizations need a fresh approach to securing these environments and will need to look beyond the traditional tools they have in their tech stack today.

    Above all, developers and security teams will need to work together to ensure security is built-in from the start. Otherwise, serverless functions will quickly become a playground for cybercriminals.

  • 9 Essential Ingredients for Modern Enterprise Applications

    9 Essential Ingredients for Modern Enterprise Applications

    Digital transformation. It’s the latest in a series of buzz phrases intended to describe how enterprises are embarking on top-to-bottom modernization of their IT environments. It’s easy to see why it’s a hot topic. Emerging technologies including cloud, containers, microservices and distributed applications hold the promise of greater flexibility, lower cost and faster release cycles. The benefits to companies that make this journey can be enormous.

    Unfortunately, making this journey can take years with today’s manual processes and antiquated tools. Not to mention that most IT budgets are at least 80 percent focused on keeping the lights on. Application modernization projects can be extremely complicated since legacy, monolithic enterprise applications are intrinsically dependent on underlying infrastructure and data stores. Cloning complex application environments, including critical production data, for upgrades or testing often can take days or weeks, slowing down cycles and contributing to lower quality releases. Even new applications are too often designed with infrastructure dependencies, trapping organizations onto specific cloud, server or container vendors. This vendor lock-in reduces choice and leaves IT organizations open to unexpected price hikes down the road.

    The truth is that the way we have designed, built, deployed and managed enterprise applications needs to change. Business-critical software is too often built in monolithic blocks that are nearly impossible to move to new infrastructure or storage options without lots of effort and time. The applications are hard to upgrade since updating even one small portion means rebuilding and retesting the entire thing. Managing the various versions of the discrete application modules and the resulting testing permutations can be quite challenging. It’s not unusual to hear from CIOs that they feel trapped in a world where they can only reasonably expect to update critical enterprise applications just twice a year at most.

    Business needs to move faster. There’s no reason why release cycles, even for enterprise applications, couldn’t be every few days instead of months. Application quality should actually be better in this world. Continuous delivery models such as DevOps should be an integral part of the process. And security should not be an afterthought; it should be built-in. To accomplish this, the future of enterprise applications must look very different from today.

    What are the essential ingredients for modern enterprise application platforms and tools in this new world? Here are a few suggestions.

    Complete Portability, Without Code Change

    Enterprise applications must be abstracted and separated from any dependency on underlying infrastructure, including sources of data. By abstracting the functions of an application into components that can run anywhere, it’s possible to then move the application to different infrastructure combinations, without changing a single line of code. It’s only through complete portability between cloud environments, container tools and servers that IT organizations will break vendor lock-in and gain the flexibility required to move their applications to vendors that offer the best combination of price, performance, reliability and features.

    On-Demand

    All enterprise applications should be on-demand, meaning the provisioning of new instances should be hands-free and fully automated. This implies tight integration with orchestration tools that most companies are already using to provision infrastructure. The key difference here is that the provisioning of storage options also needs to be fully automated. This is made possible by the fact that modern applications won’t have any dependencies on the specific data stores they rely on. This on-demand nature is not restricted to technical teams but includes business users that can easily perform user acceptance testing by simply provisioning full-fledged application environments.

    Self-Healing

    Modern enterprise applications should be self-healing, which means applications not only should have measurable data points to define their health but can then react based on these metrics. They must be able to scale up (or down) based on performance characteristics up to a certain cost threshold, restart on a healthy node in case of infrastructure bottlenecks and cache or deploy closer to the user based on latent use performance.

    Composable (DevOps-friendly)

    All enterprise applications should consist of a number of building blocks: APIs, web servers with content, databases seeded with data, security policies, the operating system, network configurations, deployment configurations and application binaries. By breaking apart monolithic enterprise applications into smaller components that can then act as independent services, IT organizations gain the flexibility to upgrade and automatically deploy individual components through standard DevOps processes and orchestration tools. The services can run anywhere desired, giving flexibility of choice in infrastructure or sources of data. And developers can use a point-and-click UI to pull pre-defined components together to build larger functions of an application.

    Model, Snapshot and Clone

    Once an application is separated into its essential components, a catalog can be maintained that makes it easy to create new versions of the application. Developers, test engineers and DevOps teams can then pick and choose whatever combinations of components they need to create or clone complex application environments for testing or deployment. The state of the application environments can be snapshotted, cloned and replicated. This cloning process should be efficient and quick. This dramatically speeds up development and testing processes, which to-date have been largely manual.

    Scalable

    Scaling application and infrastructure environments automatically, on-demand, and in near real-time is one of the essential capabilities that makes modern infrastructure so valuable. Any solution that is used to develop, test, deploy and manage enterprise applications should offer these same capabilities, giving the ultimate in cost flexibility, performance and scalability. As containers are moving quickly from the lab into production environments, look for a container-based solution that complements this strategy.

    Security is Included

    Application security should not be tacked on after deployment. This slows down continuous delivery processes like DevOps and creates friction between development/ops and the security team. Instead, security should be considered an essential component of the overall application environment and treated the same as any other component and baked into the application from the start.

    Push-Button Upgrades

    Upgrading enterprise applications should be made much easier by taking a point-and-click, push-button approach through a modern UI and integration with continuous deployment tools. Once an application is blueprinted into its various components, using a modern UI to choose versions and assemble the final application becomes entirely possible, saving time and reducing manual errors.

    Single-Pane-of-Glass Application Management

    By taking an end-to-end view of the enterprise application development, testing, deployment and management process, it’s now possible to create a “single pane of glass” console that gives development and deployment teams a powerful way to manage what’s been deployed to various infrastructures in the organization. With this approach, it’s also now possible to collect invaluable utilization and cost metrics which can help an IT organization continuously improve their portfolio.

    Enterprise applications of the future are going to look very different from today’s monolithic behemoths. By leveraging new technologies and processes, IT organizations will be able to dramatically speed up their modernization efforts and release cycles, improving competitive position. The inherent cost savings of this approach will be transformational as well, as IT budgets shift toward net-new innovation, not just keeping the lights on. The future looks bright indeed.

    About the Author / Rahul Ravulur

    Rahul Ravulur is the cofounder and CEO of appOrbit, which was founded in 2014. Most recently, he was the Director of Product Management at VMware managing the Availability suite of products. Before that, Rahul managed the vCenter Server and Client set of products with a special focus on scalability and extensibility to chart the evolution of vCenter to a broad virtualization management platform. Prior to VMware, Rahul held engineering leadership roles at BMC Software and Marimba.

  • The Top 10 Reasons to Use Cassandra

    The Top 10 Reasons to Use Cassandra

    The Apache Cassandra database is the right choice of database if you are looking for scalability and high availability without compromising performance for your mission-critical applications. Additionally, Cassandra’s support for replicating across multiple data centers is best-in-class, providing lower latency for users and the peace of mind of knowing that you can survive regional outages. To that end, we’ve compiled a list of the Top 10 reasons why enterprises onboarding and deploying mission-critical applications should use Cassandra (Apache Cassandra, DataStax Enterprise).

    1) Requirement for fast writes: Easily deals with data velocity, data variety and data complexity issues

    Many of the challenges associated with next-generation cloud applications center around data volume and data velocity. Is Cassandra able to handle the speed of data coming into the system? The answer is “yes” based on the amount of data and cluster size. Not only does Cassandra come with this ability out of the box, but there are systems of data pipeline architectures being built around ingestion speed. And to top it off, it scales linearly, making it easy to determine the right amount of capacity based on data flow.

    But, there are also two often overlooked components it provides: 1) data variety and 2) data complexity. Data variety is an alternate way of saying that data coming into one database can come in different forms. An example of this would be sensor input from a heart monitor and sensor input from an IV both for the same patient. The second component, data complexity, extends the previous example. The heart monitor might report 100 metrics twice per second under normal operating circumstances and up to 125 metrics once per second while the wearer is sleeping. This means the write patterns, locations, and frequencies can vary. Cassandra handles these situations gracefully.

    2) Can handle massive data sets

    If there were any questions about whether Cassandra is capable of handling large data sets, there is no need to look any further than the companies using it. They operate at massive scale—Netflix, Hulu, Instagram, eBay, Apple, and Spotify all have Cassandra working in interesting ways as part of their offerings.

    The other way you know Cassandra is up to the challenge is in use case examples. Many organizations use it for applications where data grows in an unbounded way very quickly. These include Twitter clones, a web log analytics data warehouse and telemetry or sensor data.

    3) Homogeneous environment

    Unlike some of the legacy distributed systems, Cassandra does not require outside support for synchronization. All of the required components for basic operation are built in. Since Cassandra also operates in a peer-to-peer fashion, this means that there is no master-slave or sharding setup and that all nodes in the ring are equal. Additionally, there is only one machine type that an administrator needs to worry about.

    4) Highly fault-tolerant

    Cassandra employs many mechanisms for fault tolerance. Since it is masterless, there is no single point of failure. There is also the potential for zero downtime rolling upgrades. This is because Cassandra can support the temporary loss of multiple nodes (depending on cluster size) with negligible impact to the overall performance of the cluster.

    The safety net Cassandra offers extends outside of your data center as well. Cassandra allows you to replicate your data to other data centers and keep multiple copies in multiple locations. This helps satisfy many regulatory requirements in addition to being a part of an strong disaster recovery and business continuity plan.

    5) Proven success across enterprise applications and in many use cases already

    There are already many examples where Cassandra is being used effectively. Banks and other financial institutions are storing large quantities of financial data in Cassandra. Analytics companies are using Cassandra to store web analytics data. Medical companies are using Cassandra to store sensor data and other time series inputs. There are also many companies making use of Cassandra for storing internet of things (IoT) data.

    6) Ease of administration

    Cassandra is a straightforward system to administer. With Cassandra being a masterless system, all nodes in the ring are the same—a homogenous system. It’s fault-tolerant and can support the temporary loss of nodes with minimal impact to production performance. This means that nodes are easy to replace and the requirement to replace downed nodes immediately isn’t as strict.

    7) Custom tuning

    There are a lot of knobs and levers that can be turned to get Cassandra to perform optimally for your workload and environment. You can set it up to operate in a way that is consistent with your workload. For example, if you write lots of log data and read infrequently, then there are configuration tweaks to be made for write-heavy systems. If you write heavily to one data center and then do all your reading from another data center, then you can adjust the settings on a data center-by-data center basis.  This idea of tuning isn’t just available at the application level; you can also tune the JVM and Java settings, including GC and logging levels. Changes can even be made by the drivers at connection time to aid in the performance of your system.

    8) Easy to integrate core applications

    A lot of work has been done on data manipulation and parsing systems to integrate with Cassandra. For instance, the full text search engine Apache Solr has been packaged to work with Cassandra to provide full featured search capabilities to an existing Cassandra database. Apache Spark, a big data analytics engine, also has been plugged in to work on an existing database. There are entire suites of tools that can be integrated or bolted on to increase its capabilities. These include Apache Mahout, Kafka and Zipkin, to name a few. This is important because the more tools you have available to you, the more powerful your data becomes. You also have the ability to gain more insight about your data without having to build and maintain the application systems that were previously required.

    9) Excellent Monitoring Options

    Included in the system of tools referenced in No. 8 are monitoring packages. If you are a user of automated monitoring platforms such as Datadog or Netuitive, you’ll find examples of prepackaged agents to monitor the important parts of Cassandra. You can then tack on your own additions of other metrics that are important to you. This is made possible by Cassandra taking advantage of Java MBeans and exposing them to the client. You can use these to get at much of the internal information Cassandra uses to make its own decisions and decide on its own health. Datastax also offers its own monitoring and control application called Opscenter.

    10) Amazing community

    One of the best things about any piece of software is having a great community of developers and experts available to you for help or guidance. There is a huge yearly database summit put on by Datastax, the primary backer and largest contributor of code to Cassandra. It also sponsors community events all over the world so you can meet and interact with other developers around you.

    For open-source software to be successful, there needs to be an ecosystem that develops around it. In the case of Cassandra, there are consultancies, monitoring and troubleshooting systems, plugins, instrumentation systems and backup systems. That is all a set a competencies that your organization no longer needs to own and can use what the greater community has already created. There are even PaaS companies that will completely take over the management for you, leaving you just the development of your application to focus on.

    Given the sizable number of organizations and people that are a part of the ecosystem and the Cassandra community as a whole, there is no shortage of articles, documentation and people willing to help. A welcoming and helpful community isn’t always a given, but in the case of Cassandra, it’s alive and thriving. This is important because software is always about people.  The more of them that you can interact with that have shared your experiences, the better. It also will be easier to find solutions to your problems having a network of people who might have faced them before.

    There are many reasons why Cassandra could be the right tool for your application. Knowing your systems requirements, workloads and future will help you make the right choice. As you can see, if you do choose Cassandra, you are bound to be in good company.

    — Michael Colby