Before joining Lightbits, I spent time working for some of the world’s foremost webscalers. Certainly, not every company has the buying power or the in-house development capabilities of Amazon, Google, Facebook and others, but there are plenty of enterprises that want to emulate the types of infrastructures used by the major players.
Why emulate hyperscale infrastructure?
Why not? Deploying a web-scale architectural approach lets companies enjoy the capabilities of the large cloud service providers, regardless of their size. That means, for instance, collecting better metrics to make informed decisions, using low cost vanity free servers, enjoying fault tolerant infrastructure software and the ability to orchestrate scale out applications at the same time – all to gain more efficiency, more agility, and to create infrastructure that can do more than its size might otherwise allow.
So is web-scale infrastructure appropriate for the masses?
Should companies of any size be worried about creating, rolling out and managing infrastructure at any scale? For the most part, the answer is yes. For companies with even 500 servers, having web-scale capabilities built into their IT system can be a great means for achieving goals and cutting costs. However, it’s been difficult for storage, until now.
Building a web-scale infrastructure is made possible, in large part, by open source solutions. Full disclosure – I was a co-founder of the Open Compute Project Foundation, a group that’s been instrumental in allowing companies to share and learn about efficient infrastructure. When companies combine open source operating systems with efficient hardware purchased directly from the manufacturer they can realize significant savings. Nearly every company running at scale today (Microsoft is a notable exception) runs on Linux. As the Linux kernel became more stable and reliable, it continued opening doors for companies to deploy at scale without having to pay licenses for every server.
The Best of Both Worlds – The Hybrid Approach
It all makes good sense. Lots of companies rely on and trust their web-scale needs to Amazon Web Services (AWS), Azure or other public clouds, and those huge providers are not going away. But most enterprises can enjoy the best of both worlds: using public clouds for some workloads and bringing the rest in-house to their private cloud that gives them the same sort of flexible, resilient, on-demand infrastructure without being tied to one provider. The hybrid approach can mean big cost savings along with more autonomy.
That brings me to storage. This is the one area that’s largely lagged in terms of private clouds, with few or no large impactful projects that have allowed people to deploy at scale yet. Ceph storage is the closest, but for a variety of reasons it’s not performing well enough today for high performance workloads. Solutions from many of the other large storage companies can work for scale-out deployments, but they tend to be expensive, more difficult to adopt, and they aren’t open source. Until now, that’s left companies that want to deploy fast storage at scale with the choice of either developing their own solution or buying an expensive solution.
And, back to the large webscalers – they have the resources to develop their own. The rest of the companies don’t have tens or even hundreds of developers on their payroll to do this kind of work, so they are forced to purchase solutions that just might not meet their needs.
So what’s the answer?
Well, I recently joined Lightbits Labs as their Chief Technology Evangelist for a reason: the Lightbits technology. Lightbits is the only company offering a true, composable, disaggregated storage that performs like local flash. Companies that want to deploy at scale and enjoy those efficiencies, and that want to control their own destinies in terms of cost really need to look at Lightbits; it’s the only legitimate option because it will run on any server and over standard open source NVMe/TCP.
I’ll be talking lots more about Lightbits over the coming months in this space and in person. If you’d like to learn more, discuss scale-out solutions or more, watch my latest webinar, Emulating Hyperscale Infrastructure: Lessons from Facebook, Google and Salesforce or, feel free to reach out to me via:
amichael@lightbitslabs.com
https://twitter.com/DigiAmir
https://www.instagram.com/biterrors/
