Implementing Software-Defined Storage (SDS) is essentially about decoupling your storage software (the brains) from the underlying hardware (the brawn). [To learn more about Software-Defined Storage, read: A Comprehensive Guide to Enterprise Software-Defined Storage Technology] Instead of buying a proprietary “black box” from a storage vendor, you’re building a flexible layer that manages data across any commodity hardware. Here is a simple roadmap to getting an SDS environment off the ground.
- Choose Your Architecture Style: Before touching any hardware, you need to decide how your storage will interface with your servers.
- Hyper-converged Infrastructure (HCI): Storage and compute live on the same nodes. Good for simplicity and scaling out (e.g., vSAN).
- Standalone SDS: The storage layer runs on dedicated commodity servers and serves data to a separate compute cluster over a network. (e.g., Lightbits software-defined storage)
- Container Storage: Designed specifically for Kubernetes (e.g., OpenEBS and Lightbits persistent storage for containers).
- Determine the Data Plane: Even though it’s “software-defined,” the underlying hardware will affect performance.
- Nodes: Standard x86 servers are the gold standard.
- Media: Mix NVMe/SSDs for a caching tier and HDDs for a high-capacity tier.
- Networking: This is the most common bottleneck. Ensure a high-speed, low-latency backbone with NVMe over TCP.
- Select the SDS Software: Your choice depends on your level of expertise and available budget:
- Open Source: Ceph Storage is scalable but has a steep learning curve and operational tax [To learn more about Ceph’s Hidden Tax, read: Ceph’s Hidden Tax: Operational Complexity vs. a Leaner NVMe/TCP Stack]
- Commercial: Solutions like Lightbits software-defined storage offer better performance, hardware efficiency to lower costs, and enterprise support.
- Cloud-Native: If you are 100% on the public cloud, consider AWS EBS or Azure Stack HCI.
- Implementation Steps
- Abstraction & Pooling: Install your chosen SDS software across your nodes. The software will “claim” the local disks and aggregate them into a single virtual storage pool.
- Define Policies: This is where the magic of SDS happens. Instead of manual provisioning, you create policies:
- Redundancy
- Tiering: Assign Tier 1 (All-Flash) to high-performance workloads, such as databases, and Tier 2 (SATA) to backups.
- Provisioning: Create virtual volumes. Since the storage is software-managed, you can “thin provision” these—meaning you can tell a VM it has 1TB of space, but it only consumes the physical bits it actually uses.
