Fun fact: Kubernetes is the Greek word for “helmsman.” The open-source Kubernetes platform acts as a helmsman for automating the deployment, scaling, and management of containerized applications. This orchestration capability helps enterprises group containers that make up applications into logical units called pods. Those logical units can then be managed in various deployments, locations, and models and the capabilities and popularity of Kubernetes are growing quickly. CSI plugins are essential for making persistent storage for containers manageable and portable within Kubernetes clusters, enabling a wide variety of cloud-native storage solutions, like Lightbits software-defined storage.
Kubernetes CSI Plugin
A Kubernetes CSI (Container Storage Interface) plugin is an implementation of the CSI specification that allows vendors to expose their storage systems to Kubernetes in a standardized way. It decouples Kubernetes from underlying storage systems, enabling dynamic provisioning, management, and use of storage resources without building custom storage solutions into Kubernetes itself.
The CSI plugin lets Kubernetes interface with different storage systems, like Lightbits block storage, through a set of standardized APIs. This simplifies storage management across platforms and enables easy scaling and automation of persistent storage in containerized environments.
Key Functions of a Kubernetes CSI Plugin:
- Provisioning Storage: Dynamically create storage volumes for workloads.
- Mounting/Unmounting: Attach and detach storage to/from Kubernetes pods.
- Snapshotting: Take snapshots or backups of data.
- Resizing Volumes: Resize storage volumes as needed.
Open Source Containers
A container image is a lightweight, standalone, executable package of software that includes everything needed to run an application: code, runtime, system tools, system libraries and settings. Containers are based on open-source solutions and are free to use, like OpenShift. Unlike virtual machines (VMs), containers don’t require a full operating system to run; instead, many containers can share a single host operating system. Splitting an application into smaller, independent pieces is known as “containerizing” that application. For example, a web application that uses a database may have one container for the HTTP server, another container for the load balancer, and a third for the back-end database. Once containerized, these “micro-services” can be scaled independently, helping enterprises operate more efficiently. This promotes scalable, portable, and flexible designs.
Containers make application services portable by separating them from server hardware, so they can move between hosts, data centers, and environments. If a server fails, the container instance can begin on an alternate server. Containers are also completely scalable: either up or down, in or out. If only part of the application needs to scale (like the back-end database) then just that portion can scale
Given their portability, it’s no wonder container popularity is climbing. Gartner research predicts that by 2025, 85% of organizations will be running 3 or more containerized applications in production, up 15% since 2023. This growth reflects the rapid adoption of containerization technologies like Kubernetes, which organizations use to modernize their IT infrastructure, improve application scalability, and accelerate software delivery.
Bring on Kubernetes
Orchestrating containers efficiently and cost-effectively is often left to Kubernetes. As more enterprises embrace the benefits Kubernetes provides, many find the platform’s appeal goes beyond deploying simple cloud-native applications and microservices. Now, many enterprises use Kubernetes to launch machine learning (ML) and artificial intelligence (AI) applications, which are data-intensive and require high performance and high availability.
In order to meet their resource requirements and deliver the best service experiences to their customers, the best practice for data storage for these demanding applications is to use direct-attached storage (DAS), such as Local Persistent Volumes for Solid State Drives (SSDs). The problem is that this is like applying a pair of handcuffs around your application’s wrists – they’ll no longer be free to move. Technically, the application loses portability because data can no longer be moved off that physical server.
Using direct-attached storage (DAS) for applications in a containerized environment breaks the Kubernetes promise of container/pod portability. This leads enterprises to lose the flexibility they are aiming for, along with the ability to simply scale their business.
Secondarily, deploying local SSDs on every application server results in severe underutilization, as up to 85% of overall capacity goes wasted, leading to millions of dollars in wasted expenditures. The industry is addressing this problem via disaggregation – accessing storage remotely, though it may appear to be local. Protocols like NVMe-over-Fabrics (NVMe-oF) now exist, so applications that work best with local NVMe flash can achieve the same performance with disaggregated NVMe without sacrificing the portability associated with containers and pods.
Since late 2018, Kubernetes has supported Container Storage Interface(CSI) plug-ins that allow third-party support for persistent storage needs. This enables network-attached or remote storage for data-centric, cloud-native applications but can suffer in performance or data services.
Low-latency block storage technologies are most useful for highly transactional workloads such as databases or message streaming services. There are essentially three block storage technology solutions that support the CSI plug-in: direct-attached storage (DAS), storage area networks (SAN) and NVMe-oF.
NVMe-oF
With the ease of operation of SAN and the high performance of DAS, NVMe-oF solutions are making inroads at many enterprises that value flexible, high-availability storage for persistent Kubernetes storage. A key benefit of NVMe-oF solutions for cloud-native applications is disaggregation while preserving local performance. This maximizes Kubernetes functionality and eases migration without compromising performance or availability, helping enterprises lower total cost of ownership through greater operational efficiency.
The disadvantages of NVMe-oF are complexity and reliability. Systems that use NVMe/RoCE (RDMA over Converged Ethernet) require changes to network infrastructure (special NICs) and Ethernet switch settings that are foreign to the ubiquitous TCP/IP settings found in most Kubernetes environments. NVMe/TCP doesn’t require these special settings or NICs, so it’s positioned for simpler adoption, but it’s just starting to see wider adoption. When it comes to reliability – some proprietary arrays support NVMe-oF with failover but don’t follow the philosophy of standard servers and software-defined “everything.” Open-source NVMe-oF targets tend to not offer high availability or data services.
Moving Forward
The move to containerize “everything” is driving the need for persistent container storage. The applications requiring persistent volumes (such as databases, analytics, message streaming services and log processing) are often the same ones that require low-latency, high-performance storage. The industry is looking for solutions that deliver local-storage performance while preserving application portability at affordable cost. Software-defined storage solutions that use NVMe/TCP can fill this market gap, enabling independent scaling of storage and compute to maximize application value in containerized Kubernetes environments.
Additional Resources
NVMe over TCP
Kubernetes Persistent Storage
NVMe Over TCP and Edge Storage
Ceph Storage
Disaggregated Storage
