In many OpenStack environments, Glance uses a file-based backend for image storage. It works, but it keeps image storage in a file-oriented model that is separate from the block storage architecture used by virtual machines.
If you’re already running Lightbits as your Cinder backend, there’s a cleaner, more aligned approach.
Store Glance images directly on the Lightbits cluster and boot VMs from block-level clones.
Let’s break down what changes — and why it matters.
The Default Model: Glance with File Storage
By default, OpenStack Glance uses the file backend:
[glance_store]
default_backend = file
“File” doesn’t always mean local disk. In real deployments, operators often place the image directory on:
- Local controller storage
- NFS shares
- Shared POSIX filesystems
- Distributed filesystems such as CephFS
All of these are valid and widely used.
However, even with NFS or a distributed filesystem, images remain file objects. That means:
- Image data lives in a filesystem layer
- Cloning often involves copying data
- Controller I/O can become a bottleneck
- Image storage is architecturally separate from Cinder volumes
Even when highly available, it’s still a file-based model.
The Block-Native Model: Glance with Cinder on Lightbits
When Glance uses the Cinder backend instead:
- Images are stored as block volumes
- Backed directly by Lightbits
- Clones are block-level operations
- No file copy path across nodes
- Image and VM storage share the same backend
In this model, Glance becomes metadata. Lightbits stores the actual bits.
Architecturally, the path shifts from:
Glance → Filesystem (local / NFS / CephFS)
to:
Glance (metadata)
→ Cinder
→ Lightbits
→ NVMe/TCP attach
→ Nova
Instead of a separate image tier, images and VM volumes now live inside the same scalable block storage platform.
What Happens During Image Upload?
After configuring Glance to use the Cinder store, the CLI experience does not change. Below is a sample command to create an image on the Lightbits storage cluster.
openstack image create “Glance Image Alma Linux 9.6” \
–file /path/to/AlmaLinux-9-GenericCloud.qcow2 \
–disk-format qcow2 \
–container-format bare \
–public
But internally:
- A Lightbits volume is created
- The image data is written directly to that volume
- The volume is marked read-only
- Glance metadata references the block device
You can confirm it with:
openstack volume list –all-projects –long
You’ll see a volume named:
image-<IMAGE_ID>
Type: lightbits
readonly = True
Or directly on the Lightbits cluster:
lbcli list volume
Your image now lives as a Lightbits block volume.
Booting a VM from the Image
Images stored in Cinder are read-only. To launch a VM, you create a writable clone:
openstack volume create \
–image <IMAGE_ID> \
–size 20 \
–type lightbits \
vm1-boot-vol
This creates a new Lightbits volume derived from the image. Because the operation happens at the block layer, it is fast and storage-efficient.
Then boot the VM:
openstack server create \
–flavor <FLAVOR_ID> \
–volume <VOLUME_ID> \
–network private \
–key-name <KEY_PAIR> \
vm1
Notice that –image is not used.
The VM boots directly from a Lightbits volume attached via NVMe/TCP. High Availability is handled at the block/path layer: if the primary replica goes down, the secondary path takes over and I/O resumes automatically after a brief pause (~10s).
What Changed Architecturally?
Instead of:
- Filesystem-based image storage
- Separate image and volume tiers
- File copy paths during cloning
You now have:
- Unified block storage
- Image volumes and VM volumes on the same backend
- Block-level clone operations
- Reduced controller I/O dependency
There is no separate image storage tier. There is no file copy path. Everything is handled at the block layer.
Conclusion
Using the file backend for Glance — whether local disk, NFS, or a distributed filesystem — keeps images in a file-oriented storage model separate from Cinder.
Switching Glance to use Cinder aligns image storage with your block architecture.
In a Lightbits deployment, that means:
- Images live directly on NVMe/TCP-backed volumes
- Boot volumes are created as block-level clones
- Image and VM storage share the same scalable backend
- Controller filesystem pressure is reduced
Glance becomes metadata. Lightbits becomes the storage engine.
And your OpenStack architecture becomes cleaner, more unified, and easier to scale.
In the next post, I’ll walk through exactly how to configure Glance to use Lightbits and boot a VM from a block-backed image.