While there is no single storage architecture model that fits all NoSQL databases, the often recommended approach is a distributed, shared-nothing architecture using local storage (often flash-based) at each node. Direct-attached storage (DAS) would be an example of shared-nothing architecture at the storage hardware level. This model provides the desired high performance, low latency, fault tolerance, and availability required by business-critical NoSQL databases like Cassandra and MongoDB. While DAS offers significant advantages, it’s counterproductive to today’s data center climate of reduced CapEx, OpEx, and sustainability initiatives. At the same time, critical data services inherent in a shared networked software-defined storage system are missing in DAS. In this blog, I’ll illustrate that with today’s storage solutions, you can have your cake and eat it too–efficiency, data services, resilience, and yes, high performance and low latency too. Modernizing your data platform to a disaggregated, software-defined architecture natively designed with NVMe® over TCP protocol can deliver the performance and fault tolerance your NoSQL database requires, without compromising efficiency.
Why Shared-Nothing is Common for NoSQL
DAS is a prevalent model for performance-sensitive workloads, like NoSQL databases, because historically local flash, especially NVMe storage, offered significantly lower latency, higher IOPS (faster read/write operations) than traditional shared storage, like SAN or NAS, accessed over a network. This performance gap is narrowing with modern storage solutions. NoSQL databases are often designed for horizontal scalability and a shared-nothing architecture, simplifying scaling out.
Additionally, some NoSQL databases are explicitly designed for a distributed, shared-nothing system. For example, in Cassandra, each node in a cluster is responsible for a subset of the data and uses its local disk for storage. And while there are flexible deployments for MongoDB, it strongly aligns with DAS due to its replica set and sharding architectures. When data is partitioned and replication is implemented across multiple nodes, there is higher resilience. If one node fails, only a portion of the data becomes temporarily unavailable, and the system can continue to operate using the replicas on other nodes. High resiliency is vital for business-critical database workloads..
The Trade-offs of Shared Nothing Architecture
Despite its advantages, DAS has drawbacks that contradict modern data center objectives. These include stranded capacity, which leads to resource underutilization (potentially 30% to 70% or higher), increased CapEx and OpEx at scale, and higher management overhead. Furthermore, DAS lacks the advanced data management capabilities (compression, thin provisioning, snapshots, deduplication, etc.) inherent in SAN platforms.
At scale, a DAS model’s CapEx and OpEx burdens can bloat an IT budget. Managing storage across a potentially large number of independent nodes can increase management overhead. Monitoring, upgrades, and capacity planning must be performed on individual nodes.
As if bloated IT budgets weren’t painful enough, the biggest sting for those implementing DAS models is sacrificing data management capabilities. Advanced data management services (e.g., compression, thin provisioning, snapshots, deduplication, tiering, clones, replication) are not inherently available at the storage layer in a shared-nothing model. In a DAS model, the functionality would have to be at the application level, if available at all.
A New Storage Paradigm for NoSQL
The need to reduce CapEx and OpEx, improve sustainability, and minimize data center sprawl is driving a shift towards shared storage platforms like SAN for NoSQL workloads. Modern SAN systems can match the performance of local NVMe, effectively supporting demanding NoSQL workloads.
The key to this performance is high-speed interconnect technology. NVMe over Fabrics (NVMe-oF) SANs extend the performance of NVMe flash over a network, significantly improving latency and throughput compared to older protocols. NVMe over TCP (NVMe/TCP), as a subset of NVMe-oF, combines the high performance of NVMe with the ubiquity and cost-efficiency of standard Ethernet networks using TCP as its transport layer. Unlike FC, NVMe/TCP operates over ubiquitous and cost-effective Ethernet infrastructure. This architecture model ensures ultra-low latency and high throughput without custom hardware or proprietary drivers.
Why Lightbits Excels with NVMe/TCP for NoSQL
Lightbits stands out as the premier choice for disaggregated, software-defined storage, utilizing NVMe/TCP for NoSQL databases. NVMe/TCP combines the performance of NVMe with the ubiquity and cost-efficiency of standard Ethernet networks. Lightbits leverages this to provide consistent low latency and high throughput over standard TCP/IP networks, eliminating the need for specialized hardware or drivers.
Lightbits’ disaggregated, software-defined storage architecture natively designed with NVMe/TCP delivers:
- Unmatched Performance: Matches and often exceeds local NVMe performance for NoSQL workloads.
- Cost-Efficiency: Reduces CapEx and OpEx by maximizing hardware resource utilization and leveraging standard Ethernet infrastructure.
- Scalability and Flexibility: Enables independent scaling of compute and storage, adapting to the dynamic needs of NoSQL databases.
- Enhanced Data Services: Provides enterprise-grade data management capabilities (e.g., thin provisioning, clones, snapshots, replication) crucial for NoSQL deployments.
- Resilience and High Availability: Ensures data protection and continuous operation, meeting the stringent requirements of business-critical NoSQL applications.
- Sustainability: Contributes to reducing data center footprint and energy consumption.
- Reduced Vendor Lock-in: Decreases reliance on proprietary hardware and complex supply chains.
While the shared-nothing architecture with DAS has been a traditional recommendation for NoSQL databases, modern solutions like Lightbits offer significant advantages. DAS might suffice for smaller clusters, but at scale, it becomes operationally and economically burdensome.
Lightbits’ disaggregated, software-defined architecture with NVMe/TCP presents a compelling alternative, bridging the performance gap and delivering essential data services and efficiency. By adopting Lightbits, organizations can achieve the high performance and low latency their NoSQL databases demand while optimizing costs and operations and future-proofing their data infrastructure.
To learn more about supporting NoSQL workloads using modern SAN with NVMe-oF, read my blog: “NVMe Storage: A Beginner’s Guide to Lightning-Fast Data Access.”
