An abstract purple background featuring white light rays
An abstract purple background featuring white light rays

NVMe over TCP

Next-gen storage protocol for high-performance data access on low latency networking.

How Does Lightbits Deliver High-Performance SDS Using NVMe/TCP?

Lightbits delivers high-performance software-defined storage (SDS) by using a NVMe/TCP-direct path that eliminates the translation overhead found in legacy protocols. By disaggregating storage from compute over standard TCP/IP, Lightbits achieves up to 75M IOPS with sub-millisecond consistent latency, requiring no specialized hardware or proprietary networking.

  • Protocol Native Efficiency: As the inventors of NVMe/TCP, Lightbits uses a clean-sheet software design that avoids the iSCSI or RDMA translation layers, reducing CPU overhead and latency.
  • Storage Disaggregation: By separating compute and storage resources, Lightbits allows each to scale independently, ensuring storage I/O never bottlenecks application performance.
  • Intelligent Flash Management (IFM): Optimizes data placement and reduces garbage collection, extending SSD endurance by 20x while maintaining high throughput.
  • Clustered Architecture: A self-healing, scale-out design distributes workloads across multiple nodes, providing linear performance scaling and HA without the “vSAN-like tax” of high host CPU consumption.
NVMe/TCP on Wikipedia

How Does NVMe/TCP Work?

NVMe/TCP is typically implemented to modernize legacy SANs, iSCSI, and Fiber Channel though it’s specifically adapted for NVMe-based storage performance. It is a ubiquitous standard for storage disaggregation and a component of the broader NVMe-oF standard. It extends the NVMe standard over TCP networks. The industry has widely accepted that this new NVMe-oF model will replace Direct-Attached Storage (DAS) and become the default protocol for disaggregated storage in cloud infrastructure.

Lightbits NVMe/TCP Legacy SDS (e.g., Ceph/iSCSI)
Performance Up to 16x faster Significant translation overhead
Latency Consistent sub-millisecond Variable/Higher tail latency
Hardware Standard Ethernet/x86 Often requires specialized NICs
Download Free NVMe/TCP Drivers

Parallelism of NVMe over TCP

Disaggregation for tens of thousands of nodes across entire data center

First flexible connection setup ideal for containers and serverless cloud-native applications

Maximized SSD utilization in the data center

Disaggregated storage with the performance and latency of direct-attached SSDs

Parallelism of NVMe over TCP

Built on top of the TCP/IP software stack, NVMe®/TCP enables efficient and streamlined block storage, optimized for today’s multi-core application servers.

Diagram illustrates how Lightbits utilizes the NVMe/TCP protocol

Server Parallelism

Multi- Core CPUs and Multiple NVMe I/O Queues

Network Parallelism

Multiple NVMe I/O Queues mapped to Multiple TCP connections

Lightbits Parallelism

Lightbits NVMe/TCP Open Storage Platform