ParadeDB on Kubernetes with CloudNativePG: Deployment Runbook and a DiskPressure Incident.

This runbook documents the end-to-end deployment of a highly available ParadeDB (PostgreSQL 18 + pg_search) cluster on a single-node Kubernetes host, managed by the CloudNativePG operator and installed with Helm. It captures the exact commands, the outputs observed, one real incident that occurred mid-deployment, and the root-cause fix that was applied.

Every command in the deployment procedure was run against a live lab host, and the corresponding output is reproduced so the procedure can be repeated with confidence. Hostnames and IP addresses are replaced with placeholders.

What this deployment delivers

A 3-instance CloudNativePG Cluster (1 primary + 2 asynchronous streaming replicas).
Dynamic persistent storage through the Rancher Local Path Provisioner, backed by a dedicated 350 GB filesystem (/u03).
Automatic primary/replica service routing (-rw / -ro / -r) with failover-aware endpoints.
ParadeDB search and AI extensions provisioned at bootstrap: pg_search, vector, pg_ivm, postgis, fuzzystrmatch.
A resolved node-level DiskPressure incident caused by container-runtime storage exhaustion, with root cause, remediation and prevention.

Lab-grade, not production-grade. This build runs on a single Kubernetes node, has backups disabled, uses asynchronous replication, and exposes the database through a bare NodePort without TLS. It is a solid reference for learning CloudNativePG operations, but read the Production Readiness section before using it for anything that matters.

1. Solution architecture

The build starts with a bare RHEL host and ends with a self-healing PostgreSQL cluster. RHEL becomes a Kubernetes node, the CloudNativePG operator turns Kubernetes into a database platform, and the ParadeDB Helm chart turns a Helm release into a running, replicated ParadeDB cluster. The chart wraps a CloudNativePG Cluster resource with the ParadeDB image and the right bootstrap.initdb extension list, so the operator manages every pod, volume, secret and service.

Design decisions

Decision	Value chosen	Rationale
Instance count	3 (1 primary + 2 replicas)	Exercises streaming replication and automatic failover, even though all pods share one node
Storage engine	Local Path Provisioner (`rancher.io/local-path`)	No SAN, NFS or cloud CSI available; gives dynamic PVC provisioning on local disk
Storage location	`/u03/local-path`	Dedicated 350 GB disk, isolated from the OS volume group
PVC size	100Gi per instance (300Gi total)	Leaves ~42Gi headroom on a 342Gi-free filesystem for OS, images, logs and WAL
Replication mode	Asynchronous (default)	Lowest write latency for a lab; evaluate synchronous replication for production
Bootstrap method	`initdb` via Helm chart defaults	Chart creates the extensions and the application secret; no manual SQL

Request and replication data flow

#	Hop	What happens	Why
1	Client → `paradedb-nodeport` (:30432)	psql, pgAdmin or an application reaches the node on the external NodePort	The only externally reachable entry point; internal services are ClusterIP
2	NodePort / `paradedb-rw` → primary pod	Kubernetes forwards to whichever pod carries `cnpg.io/instanceRole=primary`	Selectors track the label, not a pod name, so the path survives failover
3	Primary → replicas	Each committed write is appended to WAL and streamed asynchronously to `paradedb-2` and `paradedb-3`	Keeps replicas current for read scaling and promotion
4	Primary → local WAL + heap	The write is fsynced to the primary's PGDATA before the commit returns	Primary durability does not depend on replicas under async replication
5	Replicas → `paradedb-ro` clients	Read-only clients are load-balanced across the two replicas	Offloads reporting traffic from the primary
6	Pod → PVC	Each instance mounts PGDATA from its own 100Gi PVC	Pods can be recreated without losing data
7	PVC → Local Path Provisioner → disk	Each PVC maps to a directory under `/u03/local-path`	Dynamic provisioning; no PV is created by hand

2. Environment reference

Item	Value
Hostname	`k8s-node01.example.com` (placeholder)
Node IP / API server address	`10.0.0.10` (placeholder)
CPU / memory	12 vCPU / 146 GB RAM
Operating system	Red Hat Enterprise Linux 9.7
Kubernetes	v1.32.13 (kubeadm, kubelet, kubectl), single control-plane node
Pod network CIDR	192.168.0.0/16 (must match the Calico manifest)
CNI	Calico v3.30.0
Container runtime	containerd 2.2.5 (cgroup driver: systemd)
Helm	v3.21.2
CloudNativePG operator	v1.30.0
ParadeDB Helm chart	`paradedb/paradedb` 0.16.3 (app version 0.24.1)
Database image	`paradedb/paradedb:18-v0.24.1` (PostgreSQL 18)

Disk layout

Mount point	Size	Used at baseline	Purpose
`/`	13G	9.6G (74%)	OS root; small
`/var`	6.0G	5.1G (85%)	containerd + kubelet runtime data; source of the DiskPressure incident
`/var/log`	8.0G	1.9G (24%)	System and Kubernetes logs
`/u03`	350G	8.1G (3%)	Dedicated database and container-runtime storage for this build

Every OS filesystem is small by Kubernetes-node standards. /u03 is the only volume with real headroom, which is why both PostgreSQL data and, after the incident, the container runtime were placed there.

Namespaces and objects

Namespace	Key objects	Purpose
`cnpg-system`	Deployment `cnpg-cloudnative-pg`	CloudNativePG operator controller
`local-path-storage`	Deployment `local-path-provisioner`, StorageClass `local-path`	Dynamic PVC provisioning on `/u03/local-path`
`paradedb`	Cluster `paradedb`; pods `paradedb-1/2/3`; services `paradedb-rw/ro/r`, `paradedb-nodeport`; secret `paradedb-app`	The ParadeDB cluster and its supporting objects

3. Pre-flight checklist

Confirm items 1–4 before Part A and items 5–11 before Part B. Item 10 in particular would have prevented the incident in section 5.

#	Check	Command	Expected result
1	Root or passwordless sudo	`sudo -v`	No password prompt failure
2	Package repos reachable (BaseOS, Docker CE, Kubernetes)	`dnf repolist`	Repos listed, no errors
3	Kernel modules loadable	`modprobe overlay && modprobe br_netfilter`	No error output
4	Hostname resolves, static IP assigned	`hostname -f; ip a`	FQDN matches the intended `--apiserver-advertise-address`
5	Node is Ready	`kubectl get nodes`	STATUS = Ready
6	Helm installed	`helm version`	Client version printed
7	CloudNativePG CRDs present	`kubectl get crd \| grep postgresql`	`clusters.postgresql.cnpg.io` and related CRDs
8	Operator pod healthy	`kubectl get pods -n cnpg-system`	READY 1/1, Running
9	Target disk free space	`df -h /u03`	≥ (100Gi × instances) + 15%
10	Root and /var headroom	`df -h / /var`	Several GB free; low headroom causes DiskPressure
11	No stale objects from earlier trials	`kubectl get all,pvc -n paradedb`	Empty, or only intended objects

4. Deployment procedure

Fourteen continuous steps in two parts: Part A (steps 1–7) turns a bare RHEL 9.7 host into a single-node Kubernetes cluster; Part B (steps 8–14) installs CloudNativePG, prepares storage and deploys ParadeDB.

Part A — Kubernetes platform bootstrap

Step 1 — Prepare the operating system

sudo tee /etc/modules-load.d/k8s.conf <<EOF
overlay
br_netfilter
EOF
sudo modprobe overlay
sudo modprobe br_netfilter

sudo tee /etc/sysctl.d/k8s.conf <<EOF
net.bridge.bridge-nf-call-iptables  = 1
net.bridge.bridge-nf-call-ip6tables = 1
net.ipv4.ip_forward                 = 1
EOF
sudo sysctl --system

Kubernetes networking needs the bridge and overlay modules, and bridged traffic must pass through iptables. Without br_netfilter and these three sysctls, pod traffic is silently dropped and CNI plugins such as Calico fail their health checks.

Step 2 — Install and configure the container runtime

sudo dnf remove -y containerd.io
sudo dnf config-manager --add-repo https://download.docker.com/linux/rhel/docker-ce.repo
sudo dnf install -y containerd.io
sudo mkdir -p /etc/containerd
containerd config default | sudo tee /etc/containerd/config.toml
sudo sed -i 's/SystemdCgroup = false/SystemdCgroup = true/' /etc/containerd/config.toml
sudo systemctl enable --now containerd

kubelet delegates containers to a CRI runtime. SystemdCgroup = true makes containerd's cgroup driver match kubelet's; a mismatch is one of the most common reasons a kubelet never reaches Ready.

Step 3 — Install Kubernetes components

# add the Kubernetes package repository first, then:
sudo dnf install -y kubelet kubeadm kubectl kubernetes-cni cri-tools
sudo systemctl enable kubelet

kubelet runs pods, kubeadm bootstraps the cluster, and kubectl is the admin CLI. All three are pinned to v1.32.13 to avoid version skew.

Step 4 — Initialise the control plane

sudo kubeadm init \
  --apiserver-advertise-address=10.0.0.10 \
  --pod-network-cidr=192.168.0.0/16

mkdir -p $HOME/.kube
sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
sudo chown $(id -u):$(id -g) $HOME/.kube/config

kubectl get nodes
NAME                     STATUS     ROLES           AGE   VERSION
k8s-node01.example.com   NotReady   control-plane   10s   v1.32.13

NotReady is expected here: a node never becomes Ready until a CNI plugin is installed. The pod CIDR must match Calico's range.

Step 5 — Install the Calico CNI

kubectl apply -f https://raw.githubusercontent.com/projectcalico/calico/v3.30.0/manifests/calico.yaml

kubectl get nodes
NAME                     STATUS   ROLES           AGE   VERSION
k8s-node01.example.com   Ready    control-plane   3m    v1.32.13

Calico provides pod IP allocation, routing and network policy. The node turns Ready once its daemonset is healthy.

Step 6 — Remove the control-plane taint

kubectl taint nodes --all node-role.kubernetes.io/control-plane-

On a single-node cluster there are no workers, so the taint must go for the operator and database pods to schedule at all.

Step 7 — Install Helm

curl https://raw.githubusercontent.com/helm/helm/main/scripts/get-helm-3 | bash
sudo chmod 755 /usr/local/bin/helm
helm version
version.BuildInfo{Version:"v3.21.2", ...}

Checkpoint: kubectl get nodes shows one Ready node and helm version returns without error.

Part B — CloudNativePG and ParadeDB

Step 8 — Install the CloudNativePG operator

helm repo add cnpg https://cloudnative-pg.github.io/charts
helm repo update
helm upgrade --install cnpg-cloudnative-pg cnpg/cloudnative-pg \
  -n cnpg-system --create-namespace

kubectl get pods -n cnpg-system
NAME                                   READY   STATUS    RESTARTS   AGE
cnpg-cloudnative-pg-69bd8d7466-qzbbm   1/1     Running   0          111s

kubectl get crd | grep postgresql
backups.postgresql.cnpg.io
clusters.postgresql.cnpg.io
poolers.postgresql.cnpg.io
scheduledbackups.postgresql.cnpg.io

The operator reconciles kind: Cluster objects into pods, services and secrets. Without its CRDs, step 14 fails with "no matches for kind Cluster". Wait for 1/1 Running before continuing.

Step 9 — Provision local storage on /u03

sudo mkdir -p /u03/local-path
sudo chmod 755 /u03/local-path

Leave the directory owned by root:root. The provisioner creates one sub-directory per PVC, and CloudNativePG sets the in-pod ownership itself.

Step 10 — Install and configure the Local Path Provisioner

mkdir -p ~/k8s/local-path && cd ~/k8s/local-path
curl -LO https://raw.githubusercontent.com/rancher/local-path-provisioner/master/deploy/local-path-storage.yaml

# edit the ConfigMap nodePathMap: /opt/local-path-provisioner -> /u03/local-path
vi local-path-storage.yaml
kubectl apply -f local-path-storage.yaml

kubectl get storageclass
NAME         PROVISIONER             RECLAIMPOLICY   VOLUMEBINDINGMODE      AGE
local-path   rancher.io/local-path   Delete          WaitForFirstConsumer   19s

A StorageClass points at a provisioner, not a path. Editing nodePathMap redirects every future PVC's backing directory to /u03/local-path.

Step 11 — Remove legacy manual objects

An earlier manual attempt used a hand-written StatefulSet with static PV/PVC. CloudNativePG manages its own PVCs and must not compete with leftovers.

kubectl delete statefulset paradedb -n paradedb --ignore-not-found
kubectl delete pod paradedb-0 -n paradedb --force --grace-period=0 --ignore-not-found
kubectl delete service paradedb -n paradedb --ignore-not-found
kubectl delete pvc paradedb-pvc -n paradedb --ignore-not-found
kubectl delete pv paradedb-pv --ignore-not-found

Step 12 — Add the ParadeDB Helm repository and inspect the chart

helm repo add paradedb https://paradedb.github.io/charts
helm repo update
helm search repo paradedb
NAME                CHART VERSION   APP VERSION   DESCRIPTION
paradedb/paradedb   0.16.3          0.24.1        Deploys and manages a ParadeDB CloudNativePG cl...

helm show values paradedb/paradedb > values.yaml
helm template paradedb paradedb/paradedb -n paradedb > rendered.yaml
grep -n "kind: Cluster" -A120 rendered.yaml

Render the chart before overriding anything. The rendered Cluster confirms the image (paradedb/paradedb:18-v0.24.1), the bootstrap database (paradedb), the extensions created by postInitApplicationSQL, and that credentials are generated into a chart-managed secret paradedb-app. No manually created basic-auth secret is needed.

Step 13 — Define the override values

Only storage and topology are overridden; image, bootstrap, extensions and secrets stay at chart defaults.

# my-values.yaml
cluster:
  instances: 3
  storage:
    size: 100Gi
    storageClass: local-path
  walStorage:
    enabled: false

Sizing check: 3 × 100Gi = 300Gi against 342Gi free on /u03. With the Local Path Provisioner, PVC size is a logical limit, not pre-allocated disk.

Step 14 — Deploy the cluster

kubectl get all,pvc -n paradedb        # confirm the namespace is clean
helm install paradedb paradedb/paradedb -n paradedb -f my-values.yaml

NAME: paradedb
STATUS: deployed
REVISION: 1
...
Instances       3
Backups         Disabled
Storage         100Gi
Storage Class   local-path

WARNING: Backups not enabled. Recovery will not be possible!
Do not use this configuration in production.

The chart itself flags this as non-production. Immediately after install, the first pod failed to schedule — the incident below.

5. Incident: node DiskPressure during pod scheduling

Field	Detail
Severity	High — blocked all database pod scheduling
Detected	Immediately after `helm install`
Affected component	Kubernetes scheduler / kubelet on the single node
Symptom	`paradedb-1` cycled Evicted / Pending; cluster stuck at "Waiting for the instances to become active"

Symptom

kubectl get cluster -n paradedb -w
NAME       AGE   INSTANCES   READY   STATUS
paradedb   66s   1                   Waiting for the instances to become active

kubectl describe pod paradedb-1 -n paradedb
Status: Pending
Events:
  Warning  FailedScheduling  0/1 nodes are available: 1 node(s) had untolerated
           taint {node.kubernetes.io/disk-pressure: }.

Diagnosis

kubectl describe node k8s-node01.example.com
Taints:     node.kubernetes.io/disk-pressure:NoSchedule
Conditions: DiskPressure  True  KubeletHasDiskPressure
Capacity:   ephemeral-storage: 6134Mi
Events:
  Warning  FreeDiskSpaceFailed  Failed to garbage collect required amount of images.
           Attempted to free 306355404 bytes, but only found 0 bytes.

df -h / /var
/dev/mapper/rhel-root   13G  9.6G  3.5G  74%  /
/dev/mapper/rhel-var     6G  5.1G  935M  85%  /var

sudo du -xh /var/lib --max-depth=2 | sort -h | tail -5
375M  /var/lib/etcd
870M  /var/lib/containerd/io.containerd.content.v1.content
2.5G  /var/lib/containerd/io.containerd.snapshotter.v1.overlayfs
3.3G  /var/lib/containerd
4.3G  /var/lib

Root cause: /u03 had 350 GB free for PostgreSQL data, but Kubernetes keeps container images, writable layers and runtime metadata under /var/lib, not on the PVC disk. /var was only 6 GB and already 85% full. Once containerd's overlay filesystem reached 2.5 GB pulling the operator, provisioner and ParadeDB images, kubelet crossed its eviction threshold, tainted the node, and the scheduler refused every new pod — however much space /u03 had.

Resolution — relocate the containerd data root to /u03

Pruning images is a temporary fix, so the runtime's storage root was moved permanently to the large disk.

sudo grep -E '^(root|state)' /etc/containerd/config.toml   # root = '/var/lib/containerd'

sudo systemctl stop kubelet
sudo systemctl stop containerd

sudo mkdir -p /u03/containerd
sudo rsync -aHAX /var/lib/containerd/ /u03/containerd/

sudo cp /etc/containerd/config.toml /etc/containerd/config.toml.bak
sudo sed -i 's|root = "/var/lib/containerd"|root = "/u03/containerd"|' /etc/containerd/config.toml

sudo mv /var/lib/containerd /var/lib/containerd.old        # rollback point, do not delete yet

sudo systemctl daemon-reload
sudo systemctl start containerd
sudo systemctl start kubelet

Validation

kubectl describe node k8s-node01.example.com | grep -A6 Conditions:
DiskPressure  False  KubeletHasNoDiskPressure

kubectl get pods -n paradedb -w
NAME         READY   STATUS    RESTARTS   AGE
paradedb-1   1/1     Running   0          64s
paradedb-2   1/1     Running   0          37s
paradedb-3   1/1     Running   0          19s

Once the taint cleared, CloudNativePG's reconciliation loop resumed on its own: it created the primary, waited for it to be healthy, then created both replicas and joined them via streaming replication.

Prevention for future builds

Size /var at 30–50 GB minimum on any Kubernetes node, or relocate the containerd root at install time.
Add ephemeral-storage and DiskPressure checks to the pre-flight checklist.
After a full patch cycle on the new location, remove the rollback copy: sudo rm -rf /var/lib/containerd.old.

6. Post-deployment validation

kubectl get cluster -n paradedb
kubectl get pods,pvc,svc -n paradedb
NAME          TYPE        CLUSTER-IP    PORT(S)
paradedb-r    ClusterIP   10.96.0.21    5432/TCP
paradedb-ro   ClusterIP   10.96.0.22    5432/TCP
paradedb-rw   ClusterIP   10.96.0.23    5432/TCP

Extensions provisioned by bootstrap:

paradedb=# SELECT extname FROM pg_extension ORDER BY extname;
fuzzystrmatch, pg_ivm, pg_search, plpgsql, postgis,
postgis_tiger_geocoder, postgis_topology, vector       (8 rows)

Streaming replication:

paradedb=# SELECT application_name, client_addr, state, sync_state FROM pg_stat_replication;
 application_name | client_addr  |   state   | sync_state
------------------+--------------+-----------+------------
 paradedb-2       | 192.168.1.11 | streaming | async
 paradedb-3       | 192.168.1.12 | streaming | async

Check	Expected	Result
`kubectl get cluster`	Cluster in healthy state, 3/3 ready	PASS
`kubectl get pods -n paradedb`	3 pods, 1/1 Running	PASS
Node DiskPressure	False	PASS (after remediation)
`pg_extension`	8 extensions incl. `pg_search`, `vector`	PASS
`pg_stat_replication`	2 replicas streaming	PASS
`/u03/local-path`	3 PVC directories	PASS

7. External connectivity

The built-in services are ClusterIP only. For clients outside Kubernetes (pgAdmin, DBeaver, app servers), a separate NodePort service is created — never edit the operator-managed paradedb-rw, which CloudNativePG overwrites on reconciliation.

Method	Best for	Notes
`kubectl port-forward`	Ad-hoc testing	Stops when the terminal closes
NodePort (used here)	Single-node labs, small on-prem	Static host:port, tied to one node's IP
LoadBalancer	AKS / EKS / GKE	Needs a cloud load-balancer controller
External TCP LB (F5 / HAProxy / NGINX)	Production, multi-node	Recommended target architecture

# nodeport.yaml — selector copied from paradedb-rw so it follows the primary
apiVersion: v1
kind: Service
metadata:
  name: paradedb-nodeport
  namespace: paradedb
spec:
  type: NodePort
  selector:
    cnpg.io/cluster: paradedb
    cnpg.io/instanceRole: primary
  ports:
    - name: postgres
      protocol: TCP
      port: 5432
      targetPort: 5432
      nodePort: 30432

kubectl apply -f nodeport.yaml
kubectl get secret paradedb-app -n paradedb -o jsonpath='{.data.username}' | base64 -d
kubectl get secret paradedb-app -n paradedb -o jsonpath='{.data.password}' | base64 -d

Connect to <node-ip>:30432, database paradedb, with the credentials from the paradedb-app secret. Never hard-code them in client configs.

Warning: NodePort 30432 is reachable by anything that can route to the node, with no TLS and no network policy. Treat it as a temporary lab access path.

8. Services and failover behaviour

Service	Routes to	Typical use
`paradedb-rw`	Current primary only	Writes and default application connections
`paradedb-ro`	Replicas only	Read-only and reporting workloads
`paradedb-r`	Any instance	Load-balanced reads where slight staleness is fine
`paradedb-nodeport`	Current primary	External client access

If the primary becomes unreachable, CloudNativePG promotes the most caught-up replica and moves the cnpg.io/instanceRole=primary label to it. Every client-facing service selects on that label, so connections follow the new primary with no DNS or application change.

# controlled failover test (safe in a lab)
kubectl get endpoints -n paradedb -w          # keep open in a second terminal
kubectl delete pod paradedb-1 -n paradedb
# the rw/nodeport endpoint switches to the promoted replica within seconds;
# the deleted pod is recreated and rejoins as a replica

On a single node this validates the operator's failover mechanism, not resilience to a hardware or node failure.

9. Production readiness assessment

Close these gaps, roughly in this order, before any workload with real availability, durability or security needs.

Priority	Gap	Current state	Recommended action
1 — Critical	No backups	Backups disabled	Configure barman-cloud or volume-snapshot backups with WAL archiving, and test a restore
2 — Critical	Single Kubernetes node	All 3 instances on one host	At least 3 worker nodes with pod anti-affinity
3 — High	Async replication	`sync_state = async`	Evaluate synchronous replication for zero data loss
4 — High	Unencrypted external access	NodePort, no TLS, no network policy	TLS, source-IP restriction via NetworkPolicy or firewall, a proper load balancer
5 — Medium	No resource requests/limits	Chart defaults	Size CPU and memory requests/limits to the workload
6 — Medium	No monitoring	No Prometheus/Grafana	Enable the CloudNativePG metrics endpoint; alert on replication lag and disk
7 — Medium	Undersized OS partitions	`/`, `/var` under 15 GB	Apply the `/var` sizing guidance before installing Kubernetes
8 — Low	Hand-maintained NodePort	Created outside Helm values	Manage it as a Helm `services` override

Treat priorities 1 and 2 as blocking for any non-lab use; close 3–6 before external users depend on the database.

10. Troubleshooting quick reference

Symptom	Likely cause	First commands
Pod Pending / Evicted	DiskPressure, MemoryPressure, low ephemeral storage	`kubectl describe node`; `kubectl describe pod -n paradedb`
Cluster stuck "Waiting for instances"	Primary pod not scheduled or ready	`kubectl get pods -n paradedb`; `kubectl describe cluster paradedb -n paradedb`
PVC Pending	StorageClass missing or provisioner down	`kubectl get storageclass`; `kubectl get pods -n local-path-storage`
YAML "did not find expected key"	Hidden characters or CRLF line endings	`cat -A file.yaml`; `dos2unix file.yaml`
helm install OK but no Cluster	Chart values schema differs from assumptions	`helm template ... > rendered.yaml` and inspect
Cannot connect on NodePort	Selector mismatch, firewall, no labelled primary	`kubectl get endpoints -n paradedb`; `kubectl describe svc paradedb-nodeport -n paradedb`
Replica not "streaming"	Network issue, WAL gap, still catching up	`SELECT * FROM pg_stat_replication;`; `kubectl logs -n paradedb`
"Failed to garbage collect required amount of images"	containerd storage exhausted	`df -h /var`; `sudo du -xh /var/lib --max-depth=2 \| sort -h` — then relocate containerd

11. Rollback and cleanup

# uninstall the ParadeDB cluster
helm uninstall paradedb -n paradedb
kubectl delete svc paradedb-nodeport -n paradedb --ignore-not-found
kubectl delete namespace paradedb

# full teardown
kubectl delete -f local-path-storage.yaml
helm uninstall cnpg-cloudnative-pg -n cnpg-system
kubectl delete namespace cnpg-system
sudo rm -rf /u03/local-path        # only once the data is no longer needed

Local-path PVs use a Delete reclaim policy, so their directories are removed with the PVC — confirm with a listing before assuming the space is reclaimed. Rolling back the containerd relocation restores the 6 GB ceiling and will likely reproduce the incident, so only do it if the relocation itself is suspected.