If your cluster uses containerd, ctr provides direct access to namespaces and snapshots.
ctr namespace ls # List namespaces (e.g., k8s.io) ctr -n k8s.io snapshot ls # Show all snapshots (image layers) ctr -n k8s.io snapshot mount <key> /mnt # Mount a snapshot to inspect Snapshots are immutable directories linked together via overlayfs. Each snapshot has a "parent" link to the previous layer. 3. crio-status – CRI-O’s Inspection Tool For CRI-O users, crio-status dumps storage and runtime information.
Master these tools. Respect the link. Debug with confidence. Have a specific CRI filesystem issue related to links? Use the commands above to inspect your environment, and always test link operations in a non-production cluster first. cri file system tools link
# Get container PID crictl inspect <container> | grep pid nsenter -t <pid> -m bash Inside, check for broken symlinks find / -type l -xtype l 2>/dev/null
systemctl stop containerd mv /var/lib/containerd /mnt/new-disk/containerd ln -s /mnt/new-disk/containerd /var/lib/containerd systemctl start containerd Ensure the link is absolute and permissions (owner root:root , mode 0755 ) match. Advanced: Manipulating CRI Snapshots with Hard Links for Fast Cloning Hard links are not just for files—they can be used at the directory level (via cp -al ) to create instant clones of container root filesystems without copying data. This is a powerful technique when you need multiple copies of a snapshot for testing. If your cluster uses containerd, ctr provides direct
ctr -n k8s.io snapshot rm <snapshot-key> ctr -n k8s.io snapshot gc # Garbage collects unlinked snapshots Check /var/lib/containerd/io.containerd.snapshotter.v1.overlayfs/metadata.db (a BoltDB file) for orphaned links. Tools like boltdb-viewer can inspect it. Scenario 3: Migrating container rootfs to another disk using symlinks Suppose your /var/lib/containerd partition is full. You can move the storage directory and create a symbolic link.
Every time you run a container, remember: that root filesystem is an elegant chain of links. When a container starts, the runtime resolves a series of snapshots, binds them with overlayfs, and presents a unified tree. When storage fails, it is often a broken or misdirected link. Respect the link
Rebuild the image ensuring absolute symlinks or correct relative paths. Scenario 2: Disk space leak from dangling snapshot links Sometimes, the parent link remains even after the child snapshot is deleted, preventing garbage collection.