Tag: Data Recovery

  • Enlisting the C-Suite in Disaster Recovery Initiatives

    Enlisting the C-Suite in Disaster Recovery Initiatives

    When disasters make headlines, corporate executives often have knee-jerk reactions and call for business continuity solutions to be implemented immediately. However, when push comes to shove, those very same executives tend to put up roadblocks with concerns over budget, priorities, vulnerability assessments and security.

    Perhaps more concerning, according to a June 2015 Ponemon Institute report, there are “a large portion of companies who remain vulnerable as a result of lack of executive interest in preparing for worst-case scenarios.” This creates a precarious situation for DevOps professionals that understand the severity and breadth of threats that can trigger downtime—from human error to power outages, cyberattacks and natural disasters.

    To expedite disaster recovery initiatives, IT teams must enlist top executives that influence budgeting directly. This often calls for varied approaches that speak to specific executive concerns and issues.

    Understand Your Options: Disaster Recovery vs. Backup

    It’s very common for companies to confuse disaster recovery and backup solutions. Both are available via cloud, giving teams access to capabilities that were once only available to enterprises with huge IT footprints. When you approach any executive, it’s critical to understand various services and how they can work in tandem to deliver the most cost-effective protection.

    Backup ensures companies do not completely lose data because it is backed up with copies. While it is important to have copies of data, accessing and restoring backups can take some time. Therefore, for workloads with important but not extremely time-sensitive data, backup can be sufficient while satisfying budget restrictions. In the fast-paced DevOps world, backup usually falls short.

    On the other hand, cloud-based disaster recovery ensures minimal—if any—downtime. Applications can fail over and be up in running in seconds. Further, testing can be done at will without any impact on production environments. For workloads that demand continuity of service and aggressive recovery point objectives (RPOs), disaster recovery solutions are the better option and can also deliver cost efficiencies.

    For many operational and regulatory compliance reasons, companies may require a mix of both backup and disaster recovery. When evaluating service providers, check into how easy it is to apply backup and disaster recovery based on your development and application requirements. Take a close look at pricing models associated with each—including any support fees.

    CEO/COO – Chief Executive Officer and/or Chief Operations Officer

    While CEOs and COOs generally delegate detailed budget decisions to their teams, they are still ultimately responsible for overall company performance and reputation. Anything that threatens ongoing conduct of the business is their purview. Be prepared to discuss risks in real terms, and clarify that a “disaster” can be triggered by something as simple as human error.  They will care most about considerations such as:

    • Time to business resumption [e.g. near-zero recovery time objective (RTO)]
    • Capital investment via pay-as-you-go options
    • Ability to prioritize data and workloads, both for budgetary and business continuity purposes
    • Data loss estimates, in the case of a disaster
    • Testing flexibility, to ensure an operational, reliable solution
    • Independent validation from similar customers or analysts that a solution is reliable, proven and cost-effective

    CIO – Chief Information Officer

    Theoretically, this should be the easiest executive to convince because the CIO has a personal stake in recovery, but remember CIOs have a tough balancing act between controlling budgets and limiting exposure. The 6 Disaster Recovery Doppelgangers quiz and e-book can help CIOs approach their disaster recovery challenges across the organization. CIOs generally focus on:

    • Ability to see and manage the recovery process
    • Scalability of resources with on-demand access, often honing in on flexible terms
    • Security and compliance capabilities and support
    • Testing at will to ensure solution performs despite configuration changes to the day-to-day operational environment

    CFO – Chief Financial Officer

    Everything boils down to financials for CFOs, so they should be approached from a fiscal angle that demonstrates value backed by metrics. Be prepared with unbiased, independent information that appeals to their risk averse nature. What other financial benefits might come from leveraging cloud-based disaster recovery? For example, in some cases DevOps teams are beginning to use the failover cloud as a load testing environment that would otherwise require significant on-premise investments. Disaster recovery-as-a-service (DRaaS) environments with advanced security can also be used to check for possible vulnerabilities to cyberthreats.

    CFOs often care about:

    • Flexible pricing options that can be tailored at a granular level based on services required
    • Pay-as-you-go pricing models that avoid upfront capital investment
    • Geographic constraints on their data, particularly if backups or failovers need to be maintained in a specific location due to jurisdiction requirements
    • Extended DR use cases that could offset other costs like security testing and quality assurance

    CMO – Chief Marketing Officer

    On top of branding, advertising and public relations responsibilities, CMOs have key objectives that pertain to customer loyalty and overall corporate reputation. They may not be interested in the details of services, but can be a powerful advocate that can speak to the need for business continuity solutions that will protect brand image, reinforce messages about your technology innovation and maintain excellent customer support. Familiarize yourself with real-world stories that demonstrate how media—both traditional and social—cover outages and their negative impact to business. One need not look further than the recent Delta and Southwest outages.

    The concept of DevOps is about culture as much as it is about technology, and communication and collaboration are key. By understanding the needs and responsibilities of all stakeholders involved, teams can cut through politics, efficiently protect business and begin to leverage cloud-based DR technologies for initiatives far beyond recovery, alone.

    About the Author / Lilac Schoenbeck

    iLand Head Shots 4/22/14

    Lilac Schoenbeck is vice president of Product Management and Marketing at iLand. She has nearly 20 years of experience with product management, marketing, strategy, business development and software engineering in the grid, virtualization and cloud domains. She is currently vice president of marketing at iland, a global enterprise cloud hosting provider. Prior to this role, she led cloud and automation marketing for BMC Software and has worked for IBM, Fortisphere, Innosight and the Globus Alliance. Lilac holds an MBA from MIT Sloan School of Management and a Computer Science degree from Pacific Lutheran University.

  • Enterprise Cassandra Deployments

    Enterprise Cassandra Deployments

    Successful Cassandra deployment in the cloud takes a certain amount of skill and knowledge. In this blog I will share best practices for a successful Cassandra deployment in the cloud such as Amazon Web Services (AWS) and Google Cloud Platform (GCP). I will also discuss how you can protect and recover the data in your Cassandra cluster when deployed in the public cloud.

    Cassandra Deployments in the Public Cloud

    There are two main ways to have successful enterprise Cassandra deployments in AWS:

    • Use a ‘managed’ Cassandra deployment. This is a reasonable approach; the only downside is the inability to customize data placement beyond what is typically provided by the vendor.
    • Manage the cluster yourself, if you want to have full control over your Cassandra deployment and customize it to scale and perform according your business needs.

    Please refer documentation from Datastax or Apache Cassandra for detailed best practices for Cassandra deployments. The following best practices are reiterated here to simplify deployment of a scalable Cassandra cluster in AWS.

    Storage

    Both AWS and GCP offer two types of local storage options for their compute instances. For AWS, the options are Elastic Block Storage (EBS) and Ephemeral Storage. In GCP, the options are Persistent Disks and Local SSD. Both options have their own trade-offs. Pick the one that best suits your deployment.

    Persistent Storage

    EBS volumes have performance characteristics similar to network storage such as a NAS or SAN. Because storage from EBS volumes is multitenant in nature, it is not suitable for distributed, log structured databases such as Cassandra. If you want to ensure reliable performance through EBS volumes, use “EBS optimized instances” with “provisioned IOPS.” Since the IO traffic to EBS volumes goes over the instance network, EBS optimized will provide reliable performance, but does not guarantee any increased performance. To get the most performance out of EBS volumes, use instances with multiple disks and separate the volumes used for “Data” and “Commitlog.”

    If you have to use EBS Volumes, go with “EBS Optimized Instances,” with “Provisioned IOPS” and split data and Commit Log on different volumes.

    Ephemeral Storage

    Ephemeral storage is a great option to use for downstream Cassandra environments such as test/dev and staging. Since ephemeral volumes are typically SSDs, there is no need to separate the data and commitlog into separate drives. To optimize performance, choose EC2 instances with multiple ephemeral storage drives and stripe them together into one logical volume. Send data and commitlog to the logical volume. Remember that when you stop the EC2 instance, the data in Ephemeral storage will be deleted.

    Stripe all Ephemeral drives into a single volume to get maximum performance.

    Data Distribution

    Cassandra uses Snitch to determine the topology of the cluster. The Snitch is also used to determine the data center and rack for each node in the cluster. Replication strategy is used to determine the placement of data across the different nodes in the cluster.

    AWS

    1. EC2Snitch works well for single region deployments.
    1. Ec2MultiRegionSnitch works well for cluster distributed across multiple AWS regions. Please note, Multi-DC, Active-Active configuration makes your cluster resilient to region failures and for multi-dc cluster, use “local quorum” to improve performance.

    Google Cloud Platform (GCP)

    1. Use GoogleCloudSnitch for Cassandra deployments in GCP.
    1. GoogleCloudSnitch treats regions as Cassandra data centers and availability zones as racks.

    Other Features

    Both AWS and GCP offer a series of operational capabilities that seem to make it easier to maintain high uptime on Apache Cassandra cluster. These capabilities have to be chosen carefully depending your deployment needs.

    1. Load Balancing: Network-level load balancing service is offered by both AWS (Elastic Load Balancer) and GCP. It is not recommended to put a load balancer in front of Cassandra. It is better to use a client connector that can handle this, consider open source utilities such Presto and Astyanax
    1. Autoscaling: Both AWS and GCP offer Autoscaling facility for compute instances. However, although this functionality looks very enticing, putting all C* nodes in an autoscaling group will not produce the desired result. Instead, create a separate autoscaling group of ‘1 node’ for every node in your cluster. This will ensure that anytime a node goes down, it is automatically brought back up. This will ensure the cluster data placement is not disturbed when nodes go down and nodes coming up do not cause excessive data movement.

    Scalable Backups and Reliable Point-in-Time Recovery

    Backing up your data is a must-have for enterprise organizations. The impact of data loss is so high that no enterprise wants to risk relying alone on replication. Protecting data in a public cloud environment may seem daunting, but there are several options available that can make this easier compared to an on-premise deployment. A simple option may look like one of the native snapshots:

    Storage Snapshots

    I have seen few customers use snapshots of EBS volumes as a way to protect the data in their cluster against logical and/or operational errors. This solution will work but with some severe limitations. On top of the overhead of maintaining the scripts to take snapshots, there is a large overhead in storage costs. Taking a snapshot of volumes does not allow for incremental backups of your data. This causes an exponential bloat in the amount of data stored in snapshots, leading to high cost of the backup solution.

    Amazon S3 or Google Cloud Storage

    Using an S3 bucket as the secondary storage enables enterprises to use low-cost, highthroughput storage, while reducing overall operating costs for storing backup data. Here are a few recommendations when using S3:

    1. S3 Region: Amazon S3 buckets are region-specific. A bucket created in one region will provide better performance characteristics to other infrastructure in the same region. Subsequently, it is important to ensure the S3 bucket used as the secondary storage is in the same region as the production database being protected and the Datos IO cluster.
    1. Storage Class and Lifecycle Policy: Amazon S3 allows users to create lifecycle policies on buckets. These policies dictate the performance and response time characteristics of objects in the bucket. STANDARD_1A is more suitable for longer lived and less frequently access data, and is not suitable as secondary storage. Additionally, do not turn on any lifecycle policies on the buckets used for the secondary storage. All buckets should be in the standard storage class.

    Conclusion

    As enterprises leverage the agility provided by public cloud environments, it’s important to understand the various facets of Cassandra deployments to ensure the efficient use of your money and time.