There are three clear trends in storage strategies today: Most smaller enterprises rely on public clouds; mammoth enterprises build their own clouds; and the rest are stuck somewhere in between.
For the companies in the middle, whether to store their company’s data in the cloud typically comes down to cost. But cost should not be the only deciding factor.
Flash and Affordability
Dramatic price drops for flash storage are bringing on-site storage costs down. Today is the “perfect storm” for a move to NVMe-based flash: With prices dropping as much as 10 percent to 20 percent quarter to quarter, more applications are using flash; flash capacity and performance are expected to continue growing; and users of flash become hopelessly addicted because flash is just that good and fast.
One-size-fits-all flash storage solutions, however, rarely fit all companies. Most flash users today are typically doing one or more of the following:
– Traditional SAN for legacy or enterprise workloads
– Building a hyper-converged infrastructure at a smaller scale than the storage giants
– Implementing storage-defined software for ease of use
– Separating storage and compute; disaggregating for cloud-native applications, cost reductions, flexibility, speed and scale
Networking technology
Faster networks mean many enterprises have access to the same technologies used by the big public cloud vendors. Network speeds continue to ramp up as storage and compute needs grow exponentially. Today, a 100 gigabit Ethernet (Gbe) network — unfathomable just a decade ago — is a commodity standard. Also upping the ante is the ready availability of smart network interface cards (sNICs).
Having these technologies in-house means companies have greater capacity to handle their growing applications demands, such as the need for large-scale, high-performance solutions with low latency, along with availability and reliability.
As a data center cost-saving measure, separating storage from compute — disaggregation — is the next big trend because separating them makes each faster. Disaggregation also better utilizes flash, while taking advantage of high-speed data links.
Clouds and Savvy Marketing
Cloud marketers are savvy, advertising great deals for companies that move their data to the cloud. CIOs may then find themselves on the hotseat with the CFO, forced to explain their reasons for not moving everything to the cloud when the CFO sees internal costs that are far higher than the advertised cloud pricing. What the cloud marketers don’t publish is the “catch,” which is that once the data is in the cloud, any thoughts of ever taking it back — even if prices rise astronomically — are slim. And companies that move to the cloud may be disappointed in their uplinks to the cloud. In fact, the move could cost them low latency, more expensive data migration (both in and out), they’ll see increased regulations, increased data privacy concerns, and they’ll be losing some control over their business critical data. Add to that reduced elasticity around applications, and less ability for storage and compute to scale on demand and it becomes clear why the customers are hyperscalers are in a conundrum.
Cede Control or Build Your Own?
For the company comprised of a couple of engineers from Berkeley building a new app, the question of whether to build their own cloud is a no-brainer. With few exceptions, companies like this should focus their efforts on their core business and rely on AWS,GCP or another cloud providers for everything else.
There also are companies for whom the cloud is not a viable option: Medical industries with sensitive patient data, companies bound by GDPR regulations, companies with extreme latency sensitivity or other security regulations may not be legally able to take advantage of public clouds. Many times, these companies will take a hybrid approach, using the cloud only for non-regulated data.
Even companies that can avail themselves of public clouds know public clouds have other serious “gotchas.” Cloud vendors will roll out the red carpet for companies making the move to their services. But once the data has all been uploaded to their clouds, it becomes almost impossible to get it back – after all, the company will no longer have the bandwidth or the storage available even if it were easy to call it back. The other catch is that being locked in to a cloud vendor can mean companies are at their mercy if prices go up because there’s just no easy way out.
For companies that lie between the small players and the Facebook of the world, working with a cloud provider may be the right solution for some aspects of their businesses. Like smaller companies, they should build what they really need, work with cloud vendors where it makes sense and retain their storage in-house.
That’s because while many companies find it easy and efficient to move their elastic computational applications to the cloud, storage is a different beast. Moving storage to the cloud or between clouds is very expensive and can expose that data to security risks. Companies that want to avoid this risk while taking control of their storage require solutions that provide hyperscale-like efficiency for their on-premise and hybrid clouds, but that do not require a hefty engineering investment.
A Balanced Approach
There is a growing focus on aggregating and pre-processing data at the edge to reduce its movement, which typically equates to private clouds that allow organizations to move to more self-service IT and service-level storage.
A better approach for high-growth businesses is to move to a private cloud for everything, or a mix of hybrid of private clouds for storage and public clouds for computational activities with the mantra of “build, buy, integrate and automate.”
– Building the applications that are proprietary to the company’s central function – those applications that are key to the business and that no one else will build – makes sense. Buy solutions that provide the hyperscale experience and offer a clear return on investment (ROI). Look for open source solutions and the commodity hardware being used by the top hyperscalers rather than being locked in to proprietary solutions. Following these trends will help solve supply chain, reliability, availability and cost issues.
– Integrate existing solutions together, including both proprietary and open source, because no single solution provides everything the enterprise needs. Think decisions through carefully: Having a very high performance, sophisticated system is great but if it can’t easily integrate with other systems, it’s likely not the right choice. Avoid islands of technology.
– Automate everything – and automate like crazy. Uniform management layer, uniform APIs, disk overrides, firmware and software upgrades, maintenance work, switch upgrades and more all can be automated, dramatically reducing operating expenses and extending the life and uptime of the systems. All of this helps drive down the operational costs of ever-expanding infrastructure and drives a better user experience for your customers..
This more measured approach helps enterprises avoid cloud vendor, hardware vendor and software vendor lock in. They save money by investing in custom hardware only when necessary. Key to this approach is disaggregating block storage because everything else can be built on top of it, and because it’s the way to get to hyperscale storage.
Emulating AWS and the Other Big Players
For those companies on the journey to hyperscale — going from tens or hundreds of systems to thousands of systems – or for those that already have thousands of systems but want to make a leap to the next level without having to move to public clouds, it’s time to rethink their next move. Before simply designing a patchwork system of storage solutions, they should look at Amazon, Google, Facebook and others and see how they built their clouds.
While working at the same scale as those mammoths is unlikely, there are plenty of lessons companies can learn by their lead.
For example, one of the top consumers of flash storage is Apple. It seems, then, that companies can wisely select the same flash used by Apple or the other giants because it’s a sure bet they’ve done their homework and made a strong choice. It also pretty much guarantees there will always be a good supply of the flash, that the solutions are high reliability, that the prices will remain competitive because they are selling in huge volumes, and more.
As these companies grow their numbers of servers, there is even more they can learn and leverage from the big players – from suppliers to supply chains, applications, technology, use cases and more.
What’s Next?
The trend in the coming years for the “in-between” enterprises is infrastructure that combines on-premise, hybrid clouds and public clouds.
For storage, emerging solutions in hyperscale storage will help companies avoid the storage trade-offs they’ve had to make between rack-scale storage (which can limit scale, despite its high performance), DAS (which under-utilizes the infrastructure despite its high performance), and traditional SAN/HCI (which offers high utilization and advanced services, but low performance).
These new solutions, built around commodity hardware and standard servers, offer DAS performance, are easy to scale and deploy, provide high utilization and rich storage services, and help companies find true TCO benefits.
In the end, moving to public clouds and its inherent risks for storage can be avoided with solutions that introduce better utilization, performance, and ease of deployment.
