devopsy.kinsta.cloud

Tag: NVMe

  • What Should Business Owners Know About SSDs?

    What Should Business Owners Know About SSDs?

    Without a doubt, SSDs have become the latest trend in hardware storage, and you can expect their use to increase over time. Implementing SSDs in your business can improve productivity and make it more efficient by increasing the speed of your PC long enough to extend its useful life. Here is all you need to know about using an SSD for your business.

    What is an SSD Storage Device?

    An SSD is a storage device that functions similarly to an HDD. But instead of having moving parts, an SSD stores data on interconnected flash memory microchips that continue to store information even when there is no power flow. These microchips are known as NAND and are more reliable and faster than the HDD.

    SSDs are fast because they use the Non-Volatile Memory Express (NVMe), which allows high data exchange speeds of up to 2,600 MB/s, almost five times faster than a standard SSD. The NVMe SSDs are more recent than SATA SSDs and use the peripheral component interconnect express (PCIe).

    SSDs Vs. HDDs

    SSDs and HDDs differ in several ways, and you should understand the fundamental differences between the two. HDDs are mechanical and use magnetic heads to read and write data while spinning the platters. The number of rotations per minute determines the speed of an HDD.

    Meanwhile, SSDs are electronic and use flash memory chips, which makes them faster than HDDs. They also use NOR or NAND flash memory as their storage medium. The NAND is smaller and faster than NOR in read/write operations and can store data when off.

    What is the Lifespan of an SSD?

    Cells wear out faster when they write more data, and a standard SSD cell has a lifespan of around 3,000 write cycles, which may seem small at first glance. But the SSD uniformly distributes the write operations across all cells, reducing the chances of the cells dying and increasing the drive’s longevity.

    SSDs also have backup cells that regenerate dead cells. For this reason, large SSDs have an extended lifespan.

    How SSDs can Help Your Business: Speed, Longevity, Energy

    SSD is ideal for businesses that handle and store vast amounts of data due to its high performance, speed, and reliability. Here are more reasons why you should use an SSD for your business:

    1. Faster Read/Write Speeds
    SSDs are faster at accessing your business data, reducing the startup time for programs and operating systems. Compared to a traditional HDD, an SSD runs up to 30 times faster, allowing you to boot your PC in seconds instead of minutes.
    2. Longevity
    An SSD may be suitable for upgrading drives in a portable PC. It has no spinning drive, which means better durability and a lower likelihood of damage or losing data when you drop your business laptop.
    3. Energy Saving
    When it comes to going green and reducing energy consumption, SSD storage devices should be at the top of your list. SSDs usually generate less heat and energy since they don’t have moving parts. Therefore, you will save substantial power when using your PC, and more battery life means more productivity.

    Challenges of an SSD Storage Device

    It has always been expensive to store enterprise-level data on servers, with SSDs being slightly more expensive. As business storage capacity increases, SSDs become more expensive, and an HDD may be suitable if you want to archive a significant amount of data. Apart from archiving data, SSDs are generally the best option for storing and transferring data.

    Bottom Line: Speed, Reliability, Improved System Cooling

    Modern businesses cannot function without storage devices, and an SSD offers a unique solution that significantly increases productivity by reducing loading times. An SSD promises faster system performance, increased reliability, and improved system cooling. Due to these advantages, your business will save both time and money.

  • Container Storage for DevOps: New Options With NVMe

    Container Storage for DevOps: New Options With NVMe

    The DevOps world is ripe for a storage paradigm change, both in how it operates within the enterprise and how it implements applications. 

    NVMe, especially NVMe-over-fabrics (NVMeoF), is both a tool and a host target that can provide such a change—especially in the new world of microservices and containerization. NVMe can drastically improve application performance over classic implementations and, given the right tools, transform the process of writing those implementations to be faster, more robust and more flexible. 

    I/O-Intensive Work

    A quick recap shows why this is so. The DevOps build toolchain, especially for large codebases, has always been heavily I/O-dependent and storage access has classically been the biggest bottleneck for building an image. The ability to cluster small, inexpensive Linux x86 servers connected to SANs twenty years ago was a first step in separating storage hardware from compute, but I/O latency was pretty awful. Build/link operations could actually take longer than the compilation of object files. 

    Virtualization helped make hardware utilization better and cheaper but didn’t really improve throughput. Cloud infrastructures such as OpenStack and CloudStack made those infrastructures simpler and more elastic to manage but, again, the IOPs bottleneck remained. 

    Containerization for DevOps made it even simpler from the applications level—just deploy the app and ramp up container instances as needed—no more dedicated infrastructure needed. And it’s cheaper for incremental functions. 

    However, not much was done to clear the I/O bottlenecks for largescale storage and increasing overall performance. 

    When the new nonvolatile memory technology of NVMe came to market in 2014 for the single server world, it delivered two game-changing capabilities. 

    1. An increase of 5x disk access I/O speed. This was, arguably, the biggest single performance improvement since multicore processors with onboard cache (think Opteron) emerged. With Linux adding support for standard, open source NVMe drivers in 2021, the options increased beyond Windows and Mac users.However, SANs and their protocol stack were still an issue. SATA- and SCSI-based SANS performance was gated to the 1.2GB limitations of those technologies and big iron to head-end the SAN still dictated high-cost infrastructure.
    2. Unlike SATA or iSCSI, NVMe has the native capability for TCP/IP or fibre channel addressing. This allows NVMeoF to be an autonomous point for data storage in a normal network. A user can now keep control of their data but use network-attached compute as a utility with roughly 6Gb speed for I/O—greatly lessening the big penalty for disk I/O.  

    Applying Containers

    The dramatic uptick in container deployments and the debut of container attached storage (CAS)—software that includes microservices-based storage controllers that are orchestrated by Kubernetes—represented another industry milestone. These storage controllers can run anywhere that Kubernetes can run, which means any cloud or even bare metal servers or on top of a traditional shared storage system. Critically, the data itself is also accessed via containers as opposed to being stored in an off-platform, shared scale-out storage system. 

    Since container-attached, cloud-native storage enables the use of small, disaggregated architecture without the need for large-scale SANs or storage controllers dovetails nicely with DevOps work. 

    Container-based applications can benefit enormously from NVMe-based storage. As experts have pointed out, since DevOps applications are all about scalability, the parallelism embedded in the NVMe spec fits nicely to support them. 

    While SANs are good for large-scale contiguous operations, they’re expensive chunks of capacity and not nimble to deploy. They’re also bound to the slower I/O storage protocols, which can negate performance gains possible by adding NVMe to that array (as opposed to discretely).

    Container-based storage with NVMe gives DevOps teams the opportunity to add storage at low cost with deterministic management of that stateful storage—in theory, of course. In practice, the temptation to add like infrastructure to current infrastructure is powerful, even with all its limitations.

    To make that theory a reality, a control plane with the fast data plane is necessary—and the market is beginning to address this need.  

    As one example, OpenEBS recently introduced an upgraded engine, MayaStor (now part of DataCore Software), which enables use of NVMeoF at impressive speed: Benchmarks with Intel SPDK architecture show a 5x read/write speed improvement over SATA.

    OpenEBS debuted a few years back as a containerized storage solution for Kubernetes-based applications. With an optimized storage engine, Kubernetes has the opportunity to perform like other, more specialized hosting and development solutions. 

    Impact for DevOps

    What does this mean for DevOps teams? Firstly, using NVMe in containerized DevOps applications speeds up the time required for large builds, thereby speeding iterations of writing software. It also removes the need for complex and expensive SANs to host build environments. This double whammy of benefits means that development is quicker and cheaper and, in turn, applications can deploy faster while using more flexible tools.

    Second, NVMe greatly mitigates I/O concerns for how application code is designed and how environments are implemented. No more being beholden to the storage trunk—NVMeoF drops the performance penalty for I/O ops and distributes traffic across a normal TCP/IP network. This is now a triple whammy—faster, cheaper and more flexible containers built atop a fast, flexible virtualized infrastructure.

    Backed by innovations in container-attached storage, DevOps teams can achieve the agility and flexibility that are paramount to its mission—and act as a proving ground for the rest of the IT team to consider alternative paths to greater efficiency and performance.

    Join us for KubeCon + CloudNativeCon Europe 2022 in Valencia, Spain (and virtual) from May 16-20—the first in-person European event in three years!

  • Dell Advances Convergence of Compute and Storage

    Dell Advances Convergence of Compute and Storage

    Dell Technologies today launched a revamped midrange storage platform that takes advantage of NVMe interfaces and solid-state drives (SSDs) to deliver 7x the performance of previous Dell EMC PowerStore platforms, in addition to allowing applications to run locally on an instance of a VMware virtual machine.

    Ben Jastrab, senior manager for storage product marketing at Dell Technologies, said the inclusion of a virtual machine in form of an AppsON capability means that rather than having to deploy applications on a server, for example, a data analytics application could be deployed directly on top of the data it needs to access on a storage system. In addition, the latest version of Dell EMC PowerStore adds support for instances of Kubernetes from VMware running on top of a virtual machine. Dell storage systems already provide support for the Container Storage Interface (CSI) needed to provide Kubernetes clusters access to external storage systems.

    Dell has also added support for Ansible and VMware vRealize Orchestrator frameworks to automate the deployment and ongoing maintenance of Dell EMC PowerStore systems. As part of that effort to automate the management of storage systems, Dell added built-in machine learning algorithms to automate labor-intensive tasks such as initial volume placement, migrations, load balancing and issue resolution. Dell EMC CloudIQ storage monitoring and analytics software makes it possible to combine machine learning and human intelligence for real-time and historical performance and capacity analysis.

    Jastrab noted that many of these innovations are made possible because the storage operating system employed within Dell EMC PowerStore has been completely revamped using microservices based on containers. That approach not only makes the operating system more flexible and resilient, but it also will afford Dell the opportunity to experiment with how it might present storage services in the future, he said.

    Automation coupled with technologies such as deduplication and compression will play a critical role in driving the total cost of storage lower in the months ahead, Jastrab said. In the wake of the economic downturn brought on by the COVID-19 pandemic, IT organizations will be under increased pressure to reduce costs at a time when both the amount and types of data that need to be analyzed and stored continue to increase, he noted, adding the Dell PowerStore system can provide access to 2.8PB of data when configured in a scale-up architecture and more than 11PB when configured using a scale-out architecture.

    IT teams will need to determine to what degree they want to rely on proprietary frameworks such as VMware vRealize to automate IT infrastructure versus an open source Ansible framework that can be applied more broadly to multiple platforms. Dell owns VMware, but Jastrab said the company is committed to providing support for multiple IT automation frameworks.

    It’s not clear to what degree applications will shift from being deployed on traditional servers to a storage system. Many next-generation applications are clearly more latency-sensitive than their predecessors, so providing more direct access to data over an NVMe interface may present IT teams with a compelling alternative. Regardless of the approach taken, the line between compute and storage continues to blur.

  • The NVMF Boogie: How Kubernetes and NVMF Will Rock

    The NVMF Boogie: How Kubernetes and NVMF Will Rock

    I recently wrote a blog about how Kubernetes creates a global computing grid that delivers unprecedented compute flexibility. Then I wrote a blog about the importance of going fully Kubernetes-native. In today’s blog I would like to drill down into the role that the NVMe over Fabrics (NVMe-oF or just NVMF) standard will play in enabling Kubernetes infrastructure flexibility.

    What is NVMe?

    NVMe is a protocol that dictates how a CPU, moves memory via the PCI bus to a storage device. NVMe communicates over a set of rings (per CPU) where commands get submitted from any CPU to the underlying NVMe device. The design of NVMe eliminates intermediate layers between the CPU and the storage device. NVMe devices consist of a controller, queues, namespaces, namespace IDs and the actual storage media with some form of an interface.

    Storage media can be grouped into sections called namespaces with an ID. In the context of NVMF, namespaces provide a way to enforce access control for the disks/consumers. Namespaces are analogous to an OS partition, except the partitioning is done in hardware by the controller and not the OS (you can still have OS partitions on namespaces). Some NVMe namespaces might be hidden from a user (e.g. for security isolation). A controller connects to a port through queues and a namespace through its namespace ID. A controller is allowed to connect to multiple namespaces and a namespace is allowed to be controlled by multiple controllers (and thus also, multiple ports).

    Imagine smearing out this NVMe device across multiple computers and you get to the next important concept, a storage fabric.

    What is NVMF?

    When you put a network between the PCI bus and the storage device, you use NVMe over Fabric (aka NVMe-oF or simply NVMF). NVMF enables fast access between hosts and storage systems over a network. Compared to iSCSI, NVMF has much lower access latency, in practice adding only a small latency difference between local and remote storage. NVMF delivers a breakthrough in throughput and seek time relative to traditional device attached storage:

    Source: Samsung.

    This is possible because NVMF uses RDMA to efficiently access the memory of another computer without using the CPU or operating system of either. RDMA is a transport protocol that bypasses the kernel stack, offloading the workload to network hardware. This means the transport layer takes place in the RDMA NIC itself, so data is transferred directly to or from application memory without having to involve the normal OS-level network stack, achieving low latency with low CPU consumption.

    Of course, NVMF performance is gated by the networking infrastructure used, whether Ethernet, Infiniband or Fibre Channel. The November 2018 final ratification of the NVMe/TCP transport specification has accelerated adoption of NVMF because it allows storage fabrics to be deployed on standard network hardware.

    NVMF can be used with white label storage systems (JBOD), feature-rich block storage systems (SAN), and storage systems built with previous generation (SATA) devices. This means data centers can simultaneously leverage low latency devices while also using more traditional storage subsystems. NVMF also supports using storage class memory as a cache in the data center compute hierarchy.

    Why is NVMF Important?

    Think of NVMe as a replacement for SCSI, and NVMF as a replacement for iSCSI and iSER. NVMF has several distinguishing technical attributes that add up to huge performance scalability at the storage infrastructure layer:

    • Protocol Efficiency: The NVMe-oF protocol is designed specifically for efficient memory transfers at scale. Designed from the ground up for scalability, the NVMe-oF protocol dramatically reduces the internal locking needed to serialize I/O and improves the efficiency of interrupt handling. NVMe-oF also supports MSI-X and interrupt steering, which prevents bottlenecking at the CPU level and enables massive scalability as systems scale. Combined with the large number of cores in modern processors, NVMe-oF delivers massive parallelism to connected devices which increases I/O throughput and lowers I/O latency.
    • Command Set Efficiency: NVMe-oF has a streamlined and simple command set that uses less than half the number of CPU instructions to process an I/O request that SAS or SATA does, providing higher IOPS per CPU instruction cycle and lower I/O latency in the host software stack. NVMe-oF also supports enterprise features such as reservations and client features such as power management, extending the improved efficiency beyond just I/O. NVMEOF devices can approach DDR4 performance levels and, with SCM, become an integral component of the computing memory hierarchy.
    • Queue Parallelism: With 65,535 parallel queues, each with a queue depth of 65,535 entries, NVMe-oF supports massive parallelism. Every CPU has a private queue to communicate with a data device and is capable of achieving high speeds because there are no locks between queues that live on separate cores. Because each controller has its own set of queues per CPU, I/O throughput increases linearly with the number of available cores.
    • Integrated Security: NVMe-oF includes a tunneling protocol that provides security features produced by the Trusted Computing Group (TCG) and other related communities. Features planned for NVMe-oF devices and systems include simple access control, data at rest protection, crypto-erase, purge-level erase and others. Additional information on NVMe-oF security is available in the Trusted Computing Group and NVM Express Joint White Paper: TCG Storage, Opal and NVMe.

    NVMe-oF is designed not only to solve the limitations of scale-out storage, but also to lay the foundation for a storage fabric that dynamically shifts as workloads demand.

    What Makes NVMF Important to Kubernetes?

    Kubernetes applications implicitly need the infrastructure flexibility enabled by NVMF. NVMF and Kubernetes makes distributed computing within sites easier.

    The key point of my Kubernetes-native blog was “(Kubernetes-native) applications break apart into their constituent elements and can deploy and scale across the latency pyramid where they fit best. The elements dance across the infrastructure following their constituents in what I imagine as a containerized ‘ballet’ focused on delivering the best possible user experience at the lowest possible latencies.”

    However, multi-cloud, multi-site and edge deployments introduce significant additional complexity for organizations operating Kubernetes at (or beyond) enterprise scale. Business needs are pushing IT teams to move workloads between sites, clouds, WAN edge and rapidly be able to clone, burst and then destroy workloads anywhere around the world as needed. Yet Kubernetes deployment across multiple regions, sites, clouds and WAN edge remains problematic.

    OpenEBS is the solution for latency sensitive stateful Kubernetes workloads that need universal data agility. A NVMF-based disaggregated storage physical fabric mates perfectly with the disaggregated data agility software infrastructure offered by OpenEBS. NVMF allows data to physically float across a single storage fabric while OpenEBS allows stateful workloads to be loosely coupled across multiple storage infrastructures. And the OpenEBS MayaStor project natively integrates to NVMF to simplify the developer experience.

    Together, NVMF and OpenEBS become the dance floor for those Kubernetes-native application components to do their infrastructure-aware (and infrastructure-intelligent) ballet. Given its performance and scalability, however, perhaps this is better called the NVMF Boogie.

    To learn more about containerized infrastructure and cloud native technologies, consider coming to KubeCon + CloudNativeCon NA, November 18-21 in San Diego.

    — Charles Piercey