devopsy.kinsta.cloud

Tag: recovery

  • How to Get the Supply Chain Back to (Better than) Normal

    How to Get the Supply Chain Back to (Better than) Normal

    If a chain is only as strong as its weakest link, today’s supply chain is in dire straits. A recent infographic from The New York Times shows all too clearly that the international supply chain is in upheaval, due directly or indirectly to the outbreak of COVID-19. At the same time, reams of data generated along the supply chain are going to waste. The internet of things (IoT) can’t single-handedly fix the supply chain, but IoT technology combined with the cloud, AI and automation has the potential to bring the supply chain (and society) back to something resembling normal—actually, something better than normal.

    Indeed, even during “normal” times, the supply chain was stretched to the limit by the challenge of constantly reducing costs and improving margins, as well as optimizing delivery times for just-in-time (JIT) inventory. With that said, the last two years have been especially grim, with factories, stores and restaurants literally shutting down for lack of materials. Then there’s the computer chip shortage, which is doing nothing less than “dragging down the economy,” according to a White House briefing room document,

    The problem is now multifaceted, so the solution is not going to be as simple as adding a better monitoring mechanism.

    At a minimum, companies need better visibility into and transparency of their supply chains. This gives them more control over the entire manufacturing process, from sourcing raw materials to delivery of final products. Technologies like IoT, AI/ML, automation and edge computing can optimize supply chain management by providing visibility and metrics so companies can make real-time decisions.

    In manufacturing, for example, IoT can bring greater visibility into the production workflow. Data from the digital twins of machinery—analyzed in real-time and fed back to control systems—can lead to improved levels of operational and business efficiency. When manufactured goods are ready to be shipped, IoT technology can guide the logistics and transportation of goods as they move from manufacturers to end users. In addition to managing supply, companies also need to get better at predicting customer demand so it’s a self-sustaining feedback loop.

    However, companies adopting these technologies need to be aware that they will not be effective on their own. As with any digital transformation initiative, people and processes are as important as technology. To ease or even overcome today’s supply chain challenges, companies must not only agree on common data exchange formats and APIs across the supply chain but also dedicate themselves to ongoing cooperation and motivation to achieve the common goal of streamlining and hardening the supply chain. To these ends, open source will be critical to the ongoing development of common, industry-wide standards that enable companies to future-proof their solutions.

    For example, carmakers such as Tesla are simplifying their supply chain by using a higher-end processor that can perform multiple functions rather than using dozens of dedicated microcontrollers for each function, with updates pushed out over the air. Organizations need to think along similar lines when it comes to their IoT architecture. Developers have historically thought about IoT applications in a discrete way—creating custom frameworks for creating apps for low-power devices. Rather, developers should be considering IoT applications as an extension of their application portfolio—using the same languages, frameworks and tools for IoT apps that they use for cloud applications.

    Post-Pandemic Supply Chain Issues

    As we move into the next phase of the pandemic, it might be tempting to believe that all will be right with the supply chain world. However, beyond COVID-19, there are several macro trends that will affect the supply chain, including climate change, cybersecurity and the changing dynamics of global trade.

    To combat climate change, for example, companies will need to reduce their Scope 3 emissions. The EPA defines Scope 3 emissions as the result of activities from assets not owned or controlled by companies themselves but by their supply chain (or value chain). Scope 3 emissions are often the majority of a company’s greenhouse gas emissions. To reach sustainability goals, companies will need better resource management of their supply chains.

    Supply chains also are increasingly being used as a cyberattack vector. This includes both physical entities, like building operations (such as the hack of Target systems via a third-party HVAC vendor) or the software supply chain (such as the SolarWinds hack). In fact, it’s no longer a matter of if an organization gets attacked, but when.

    The war on Ukraine has magnified the intricacies of our hyperconnected global economy. The U.S. government and its allies have imposed severe sanctions on Russia, a move that will have widespread repercussions. The New York Times reported that the “response that U.S. officials have promised could roil major economies, particularly those in Europe, and even threaten the stability of the global financial system.” The same could be said for the global supply chain, if Russia responds by, for example, cutting off shipments of materials needed to build semiconductors or mounting new cyberattacks.

    Conclusion

    COVID-19, a steady and stealthy rise in cyberattacks, trade barriers, war … None of these factors caused the problems we’re seeing with both physical and technological supply chains; rather, they have exposed existing weaknesses for all the world to see—and experience. Organizations must work together to increase resiliency and security through the use of key technologies such as IoT, AI/ML and edge, as well as a commitment to open, standards-based solutions.

  • When Bad Things Happen to Good Clouds: Why You Need a Recovery Plan

    When Bad Things Happen to Good Clouds: Why You Need a Recovery Plan

    Amazon S3 storage service recently experienced a widespread five-hour outage in its East Coast availability zone. Many next-generation consumer and business applications rely on a cloud storage, so the S3 outage quickly cascaded and temporarily crippled organizations from Netflix to Slack.

    Cloud outages like this are normal, and notable because they affect so many businesses. The last significant AWS outage was in August 2016. While Amazon S3 is architected for data durability, that doesn’t equal fast recoverability during an outage. As good as cloud platforms are, they still leave a few gaps:

    • Availability zones don’t equal recoverability. S3 is designed to withstand a site outage within a zone, but as the most recent outage shows, networking issues can lead to a widespread outage across an entire region.
    • Data still needs to be backed up. As we noted in a previous blog post, even Amazon recommends backing up data.
    • Recovery can be slow and tedious. It’s one thing to back up data. It’s another thing entirely to recover it. It can take hours or days to recover data after a failure—especially for hyperscale applications and databases.
    • Data is often in one “basket.” If backup data is stored in the same cloud service as the primary data, in the same availability zone, there’s no way to recover data during a widespread outage.
    • Data can get compromised or enter an inconsistent state. The cloud itself doesn’t protect data from application- or database-level corruption, or human error.

    A data backup, recovery and continuity strategy needs to be designed with the cloud in mind. To make sure you can recover quickly, even from a cloud outage:

    • Keep backup data in another service or region. Failures like this one often affect an entire region. A backup and recovery strategy needs to include the ability to recover in another region, cloud service or even a private cloud.
    • Have a fast recovery process. Traditional backup solutions and scripting-based approaches can’t recover data quickly, particularly if the application needs to be recovered to a different topology.
    • Have point-in-time recovery. Since data can get compromised at the early stages of an outage, being able to restore applications quickly to a point in time is important as well.

    The cloud as-a-service architecture is more resilient than traditional infrastructure, and gives you agility. But when failures do happen, they are also entirely out of your control. Don’t ignore recoverability and resiliency of the data because it’s in the cloud, and don’t expect the same recovery processes and tools to work on next-generation, hyperscale applications.

    — Shalabh Goyal

  • Backup and Recovery for Distributed Databases

    Backup and Recovery for Distributed Databases

    Over the past five years, applications have undergone a paradigm shift. They now require more agility, scalability and availability. The key trends in this era are driven by the use of social platforms, mobile devices and software as a service (SaaS).

    Being able to ingest large amounts of data from various sources and process that data in real-time to deliver contextual information or business insights is a major competitive advantage. To meet these new application requirements, organizations are forced to look beyond traditional relational databases. This has resulted in a slew of new database systems that are distributed and scale-out in nature, can be deployed on commodity hardware and offer tunable consistency-versus-performance trade-offs. It’s important to note that to cater to the agility requirements, database-as-a-service model in cloud is being adopted as well.

    To show just how big the shift to distributed databases is, below is a Keyword Search Trend graph showing the sharp rise in NoSQL search.

    screen-shot-2016-09-27-at-2-03-59-pmThe implication of this is that requirements for data protection have changed forever, driven by this data-centric world (web-scale applications and distributed databases). Organizations are collecting vast amounts of data and seek to drive more value from that data to make better and faster business decisions. Most distributed and cloud databases already provide replication capability and thus satisfy the availability requirements of data protection. However, data protection requirements for scalable point-in-time backup and recovery need to be addressed. Without point-in-time backups, organizations are at substantial risk of losing data due to human error, logical corruption and other operational failures. Traditional backup solutions were built to address the requirements of relational databases that used shared storage and had the ACID transaction model. Unfortunately, they fall well short in addressing the point-in-time backup requirements of distributed databases (local storage, eventual consistency and the elastic nature of infrastructure).

    As database architecture has fundamentally changed to meet new application requirements, data protection needs to be redefined and rearchitected as well. The following are the new requirements for data protection.

    • Take a durable point-in-time backup copy of an eventually consistent database; we call it “versioning” the new paradigm of data protection for this new distributed era: backups to snapshots to replication to copy data management to versioning
    • Minimize time to recover from failures (low RTO)
    • Scale with the growing needs of application
    • Enable easy refresh of test/dev environments for continuous development
    • Provide operational resiliency in event of failures
    • Provide deployment flexibility in public cloud or on-premises data center

    Most organizations are investing in enterprise-grade point-in-time backup and recovery products so they may deploy and scale their next-generation application on distributed databases with confidence. In the next five years, it will be exciting to watch companies redefine the data protection technologies to meet the requirements of next-generation applications.

    — Jeannie Liou

  • Backup and Recovery of MongoDB, Part 2

    Backup and Recovery of MongoDB, Part 2

    This blog is the second in a two-part series on backup and recovery for MongoDB. In one of my previous blogs, I covered why companies require both replication and backup for enterprise grade data protection. And in the first part of this blog series, I discussed the existing solutions for backup and recovery of MongoDB and their drawbacks. Now in this blog, I will discuss the key requirements for protecting data that resides on MongoDB (deployed either on-premises, on private cloud with an as-a-service model or in public cloud with Amazon AWS or Google Cloud Platform).

    Requirement 1: Online Cluster-Consistent Backups

    Next-generation applications deployed on MongoDB must be always-on. This means pausing the database to make backups is not feasible and the backup operation should not impact the performance of the application. As the application scales, the underlying MongoDB also needs to scale-out to multiple shards. In this case, a backup solution must provide a consistent backup copy across shards without disrupting database and application performance during the backup.

    Advanced Data Management ServicesRequirement 2: Flexible Backup Options

    Depending on the application, data may have different change rate and patterns. For example, in a product catalog, certain items may be refreshed every day (fast-selling goods, for example), while the others may have longer shelf life, such as premium items. Based on the application requirements, some collections may need to be backed up every hour versus others that may be backed up daily. Providing this flexibility to schedule backups at any interval and at collection level granularity is another requirement we have heard from customers using MongoDB. More importantly, these backups always should be stored on the secondary storage in native formats to avoid vendor lock-in.

    Requirement 3: Scalable Recovery

    During its life cycle, data resides in multiple stages such as development, test, preproduction and production, and also may reside in multiple clouds (private cloud and public cloud). The topology of MongoDB clusters at each stage is different. For production, the application could be deployed on a sharded MongoDB cluster on-premises, but the test team might have access only to unsharded MongoDB clusters in the Amazon AWS (public cloud). Hence, the backup solution should allow multiple restore operations such as sharded to sharded (such as from 5×3 cluster to 2×3 sharded cluster) or sharded to unsharded (such as 5×3 cluster to 1×3 unsharded) across such cloud configuration.

    chart

    Requirement 4: Handling Failure

    Failures are a norm in the distributed database world. However, the backup solution should be resilient to database process failures, node failures, network failure and even logical corruption of data during backup and recovery operations. Finally, the backup solution should be able to handle failures of MongoDB config servers that store metadata for sharded clusters.

    Customers are deploying MongoDB in physical servers, private clouds and microservices such as frameworks, and in public cloud.  Backup and recovery should be seamless across these deployment example and the ease of backup and recovery deployment is a big one for MongoDB customers and I will cover this in the next blog. Stay tuned!

  • Top 5 Reasons to Back Up Your OpenStack Environment

    Top 5 Reasons to Back Up Your OpenStack Environment

    I am a big fan of lists. If you recall the movie “High Fidelity,” John Cusack’s character Rob and his co-workers—armed with an encyclopedic knowledge of all things musical—compile “top five” lists for every conceivable occasion, openly mock the tastes of people, and, every so often, sell a few records. In the spirit of a lists, here are the top five reasons to back up your OpenStack environment. We thought that it would be helpful to many.

    If you’re currently working with OpenStack clouds, or are in the process of embracing it, you need to include backup and recovery for OpenStack on your list of to-dos from the start. You may ask, “Why? I have orchestrated/automated everything, I just need to spin resources up and down.” Cloud environments are fundamentally different from the virtualization we have become used to. Virtualization relies on infrastructure for resiliency. In the cloud, distributed applications are responsible for their own resiliency, and are independent of the underlying infrastructure. Therefore, cloud apps need backup and recovery options, lest you face a complete machine, data or file meltdown.

    We’ve compiled the top five reasons to backup your OpenStack environment. Take a read and it will become clear why you must include backup in your OpenStack plans.

    1. Compliance — Compliance may seem like an issue for only certain industries—finance and health care come to mind—but nearly every organization has to deal with compliance issues. For some, it’s compliance to external regulations, while for others it’s simply complying with internal policies and best practices. Backup for OpenStack is an essential component for compliance; it ensures the availability of the data at a specific point in time within your OpenStack applications. I was recently involved in a dialogue/string on LinkedIn watching experienced OpenStackers go back and forth debating “pet vs. cattle.” “I am getting tired of it,” one person said. “This is cloud. It’s about scaling up and out … With automation and orchestration there is no need for cattle.” This conversation went on for days. Theory is great. Practice is reality. What people failed to discuss is that compliance or the business units will drive many requirements (such as backup) either today or tomorrow. End of story.
    2. DevOps — Backup is often the forgotten tool for DevOps. Containers are great to use, but the ability to test with real data is an issue. The reason to use backup is quite simple: test and development. With a copy of a production environment, you now can test upgrades and new features in your agile development process. Furthermore, if you lose a machine or your data, or you end up with corrupted files while in the middle of testing or development, your life will be much easier if you have an appropriate backup solution in place. Restoring machines, data and files is a better option than re-creating them entirely. Cookbooks and recipes are great as well, but you still need to couple it with the appropriate “ingredients.” It is like playing with a fake kitchen as a child … do you want to make breakfast with plastic pancakes and toast? Nothing beats the real thing. Yes, more resources are required for one over the other, but when something goes wrong—it is unfortunate—people point fingers.
    3. Disaster recovery plans — While the terms “backup” and “disaster recovery” often are used interchangeably, they are actually not the same thing. You might have what appears to be a great recovery environment (ideally one that mirrors your production environment for high availability), but without a comprehensive backup solution, your recovery might not be the painless experience you expect it to be. Perhaps the solution for intelligent disaster recovery is to marry backup with a Swift environment so you can recover from any point in time, anywhere in your organization.
    4. Save time — Often recovery can be a frantic process, driven by a business need to have the machine, data or files recovered as soon as possible. Or, more likely, sooner than possible. If you are trying to re-create something under that type of environment, there’s a likelihood that mistakes could be made. This becomes increasingly true the more complex and mature your cloud becomes (think about the growing number of workloads and applications). A much easier approach—one that provides you with peace of mind—is to have a comprehensive backup solution ready and available for when you need it. There’s also another reason backup can save you time: When you have to re-create the lost information, you are taking your time away from something else—and that means you are taking a big hit to your productivity. Save yourself and your organization a big headache with backup.
    5. Security and forensics — You never know when you are going to be asked to provide data or files related to a security issue or a forensics investigation. The right backup solution not only makes it easier to assist with the activity, but think how good you’ll look by being able to deliver the information the forensics team needs quickly.

     Are you currently looking into backup solutions for OpenStack? Now that you know why you need backup for OpenStack, learn more about how to backup OpenStack.