800G and Climbing: The High-Performance Ethernet Era (2023–2025)

Robert Terlizzi
Robert Terlizzi
Director of Product Marketing
December 26, 2025

This is the eleventh blog post in a 12-part series charting the storage journey — decade by decade, technology by technology — showing how the cost-per-GB cliff, networking advances such as NVMe over TCP enable high-performance data access, and software innovation got us from washing machines to the frictionless, fabric-native storage we’re building today.

For most of my career, storage vendors blamed the same things when performance fell apart: disks were too slow, controllers were overloaded, or protocols weren’t efficient enough. For a long time, that was true.

It isn’t anymore.

By the early 2020s, flash had removed media as the bottleneck. CPUs had become absurdly fast. Memory bandwidth exploded. What remained—and what finally became impossible to ignore—was the network.

Today, the modern data center is no longer compute-bound or storage-bound. It is fabric-bound.

That reality is defining everything that comes next.

When the Network Became the Workload

In legacy architectures, storage traffic was largely north-south. Servers talked to an array. Clients talked to servers. Designs assumed locality, predictability, and relatively static placement of data.

That world is gone.

Kubernetes, microservices, and distributed data platforms turned east-west traffic into the dominant pattern. AI training pipelines fan out reads across entire datasets. Analytics platforms hit metadata and data paths simultaneously. Replication, snapshots, erasure coding, and rebuilds all run continuously in the background.

In modern environments, data movement is the workload.

If your architecture assumes data lives “somewhere specific,” it collapses under these patterns. If it assumes the network is secondary, it becomes the bottleneck.

Ethernet Didn’t Just Get Faster — It Got Good

The most important story of this era isn’t just that Ethernet got faster—it’s that it became predictable at scale.

Between 2023 and 2025, 100G stopped being impressive and became normal. 200G and 400G moved into production. 800G stopped sounding theoretical. But bandwidth alone didn’t change the game.

What changed was behavior.

Cut-through switching replaced store-and-forward. ASIC pipelines became deeply optimized for latency consistency. Congestion telemetry improved. Buffers stopped behaving like black holes. NICs became multi-queue, multi-core, and genuinely intelligent.

At this point, Ethernet isn’t the “cheap alternative.” It’s the only fabric that can scale bandwidth, latency, cost, and operational simplicity simultaneously.

Fibre Channel still exists. InfiniBand still dominates HPC and large AI islands. But for general-purpose infrastructure, Ethernet isn’t just winning—it’s unavoidable.

Why Native NVMe/TCP Architectures Matter

At 100G, you can get away with architectural inefficiencies. At 400G and 800G, you can’t.

This is where the difference between native NVMe/TCP design and bolt-on gateway architectures becomes obvious.

Many systems in the market claim NVMe/TCP support, but internally they rely on protocol translation layers. NVMe comes in, gets terminated at a gateway, translated into something else, processed by a legacy control path, and then walked back down the stack. Each step adds overhead. Each abstraction adds latency. Each translation collapses parallelism.

Those costs were hidden when the network was slow. They are fatal when the network is fast.

A native NVMe/TCP architecture removes those translation points entirely. There is no gateway layer. No protocol shim. No internal conversion tax. The server connects using NVMe semantics directly to flash blocks that are managed natively as NVMe—end to end.

That distinction determines whether a system scales cleanly or falls over under real load.

NVMe Queues Per Core: Parallelism That Actually Scales

For decades, storage performance eventually serialized. Shared queues, global locks, and controller choke points meant that adding hardware often added contention.

NVMe changed that.

NVMe was designed around queues per CPU core, not shared command queues guarded by locks. Each core gets its own submission and completion queues. Context switching disappears. Contention evaporates.

Performance scales with cores, not just devices. Add CPU capacity, and you add independent I/O lanes. Add network bandwidth, and those lanes remain independent instead of collapsing into bottlenecks.

When NVMe/TCP is implemented natively, those per-core queues extend cleanly across the fabric. I/O does not get funneled through a single choke point. Metadata operations don’t block unrelated requests. Mixed workloads don’t poison each other.

The result is consistency.

Small environments benefit because modest core counts can fully saturate available media. Large environments benefit from adding hosts, cores, and bandwidth, which delivers predictable, linear scaling rather than diminishing returns.

This is exactly what modern infrastructure demands.

Server-Side Access Matters Again

Modern applications generate massive parallel I/O at the host. If an architecture forces those requests through a narrow gateway or serialized control plane, performance collapses.

Native NVMe/TCP allows servers to interact with disaggregated storage the same way they interact with local NVMe—just across the network. The host keeps its parallel submission model. The network becomes an extension of PCIe, not a translation boundary.

That’s why native designs don’t just benchmark well—they behave well. Latency stays predictable. Tail behavior stays stable. Performance doesn’t collapse when access patterns get ugly.

In an era where AI and analytics can read everything at once, predictability matters more than peak numbers.

What Enterprises Actually Need Now

After decades of evolution, the requirements are no longer complicated.

Enterprises need storage that is cheap, fast, reliable, and scalable—all at once.

Security is table stakes and increasingly lives in cloud-native control planes and network layers, not in proprietary array features that break the moment modern workflows are introduced.

Performance is king.

Latency isn’t just a metric anymore—latency is death. It kills application performance, AI pipelines, and business velocity long before anyone files a ticket.

Lock-in is just as dangerous. Proprietary fabrics and closed management planes eventually explode the cost model. What works at 100 TB becomes painful at a petabyte and catastrophic at scale.

The winners in this era are platforms that deliver consistent low latency across unpredictable I/O patterns, linear scale without architectural gymnastics, efficient use of standard network fabrics, simple API-driven operations, and the freedom to evolve without ripping out the foundation.

Why 800G Is a Signal, Not a Destination

800G Ethernet isn’t the end of the road. It’s proof of direction.

1.6T will arrive. Faster optics will follow. NICs will get smarter. Offloads will improve.

The real question isn’t how fast the fabric can get—it’s how efficiently the system can use it.

Architectures that respect the fabric, align with the server’s I/O model, and eliminate unnecessary translation layers will continue to scale. Everything else will hit a wall.

Closing Thought

Every major storage transition over the last 70 years has reduced friction and increased abstraction. From mainframes to DAS, SAN to NAS, cloud to Kubernetes—the direction has always been the same.

The network is now the abstraction layer that matters most. And in a world defined by massive parallelism, unpredictable workloads, and relentless scale, native NVMe/TCP is how storage finally keeps up with the rest of the system.

To learn more, read more blogs from this series:

About the writer
Robert Terlizzi
Robert Terlizzi
Director of Product Marketing
LinkedIn