When you need to deploy the in-memory database Valkey in Kubernetes, engineers usually choose between two extremes. The first path is renting a managed DBMS in the cloud: you get fault tolerance out of the box, but you overpay for memory and lose access to fine-grained tuning. The second path is running the database inside your own cluster via a StatefulSet, surrounding it with replica failover scripts, and flinching at midnight alerts whenever the primary restarts.
I was drawn to the open-source wellcake project under the Apache-2.0 license, which was highlighted in the DevOps&SRE Library channel. Its authors eliminated the choice between expensive cloud rentals and brittle manifests: the Go-based valkey-operator controller moves Valkey operational rules—quorum control, leader failover, resharding, and backups—directly into the Kubernetes reconciliation loop. Let's look at why basic orchestrator primitives fall short for in-memory databases and how this automation works.
Why a Basic StatefulSet Is Blind to Valkey Failures and How the Operator Works
After the Redis license change, the industry rallied behind the open Valkey fork under the Linux Foundation (other RESP-compatible solutions are evolving in parallel—we recently covered KiviDB as a multithreaded Rust alternative to Redis and Valkey). Yet the old operational pains moved into Kubernetes right alongside the database.
You might ask: what is wrong with a standard StatefulSet if it gives pods stable names and disks? To the core Kubernetes controller, a database container looks no different from a web server. It is like a building superintendent who checks whether the lights are on in each office, without knowing who holds the master ledger. If the write node goes down, the orchestrator spins up a fresh container with the same network name, while replicas without an election command keep waiting for the old leader or sync with an empty node, wiping data.
To make the cluster understand database logic, you use the operator pattern—a controller with Custom Resource Definitions (CRDs). In wellcake, there are two: ValkeyCluster defines the topology and disks, while ValkeyACL configures user permissions via the idempotent ACL SETUSER ... reset command.
The controller blocks common mistakes out of the box: it forbids temporary emptyDir folders and requires PersistentVolumeClaim volumes so that kubelet does not evict a database pod when disk space runs low. The maxmemory limit is automatically set to 60% of the container's memory limit (resources.limits.memory). Instead of manually tuning dozens of parameters, two profiles are available:
Cache— a fast cache withallkeys-lrueviction, no disk persistence, and a1Givolume;Durable— a store with anoevictionpolicy,RDBsnapshots, anAOFlog, and a10Givolume.
Four Topologies in One Resource: From a Single Node to a Sharded Cluster
The key design win of wellcake is unifying four Valkey deployment schemes inside the ValkeyCluster resource. The operator is installed from the oci://ghcr.io/melancholictheory/charts/valkey-operator registry into the valkey-system namespace.
Let's walk through an end-to-end scenario: we will spin up a fault-tolerant cache in the apps namespace via the valkey-cluster chart with topology=Replication, profile=Cache, and replicas=3. Depending on the topology, the controller activates its own protection mechanics:
Standalone. A single pod without replication for test environments and ephemeral caches.Replication. The setup from our example: a primary node (primary) and two replicas connected viaREPLICAOF. Every fifteen seconds, the operator runs a TCP probe on the nodes. If the primary goes silent, the controller compares replication offsets (offset), promotes the replica with the freshest data to leader, and protects the database from split-brain (split-brain) whenpod-0returns. At the same time, the operator updates thevalkey.wellcake.io/rolelabel: the<cluster>-primaryservice routes writes strictly to the leader, while<cluster>-replicasdistributes reads across replicas.Sentinel. For applications whose clients query Sentinel directly, the operator runs a separateStatefulSetand service on port26379(26380with TLS), where a quorum of arbiters decides leader changes.Cluster. Keys are split into hash slots across masters linked by thegossipprotocol. The operator assembles the cluster viavalkey-cli --cluster createand automatically rebalances slots when scaling (autoReshard). On Valkey 9.1+, Atomic Slot Migration (ASM,ADR 0001) transfers a slot as a single transaction with no stuck keys on network failure. The operator also rescues the cluster if more than half of the masters are lost at once (ADR 0006) andgossiploses quorum: it isolates dead nodes and executesCLUSTER FAILOVER TAKEOVERon surviving replicas (automatically forCache, or via thevalkey.wellcake.io/quorum-takeoverannotation forDurable).
Seamless Operations: Proactive Rollout, Secret Rotation, and S3 Backups
Let's continue with our Replication cluster example: suppose we updated the container image or changed memory limits.
During a standard update, Kubernetes immediately terminates the primary pod: while the system detects the failure and reroutes traffic, writes stall for 15–20 seconds. In wellcake, proactive rollout (ADR 0004) is enabled with the valkey.wellcake.io/proactive-rollout: "true" annotation, switching the StatefulSet to OnDelete mode. The operator updates the replicas one by one and, before stopping the primary, hands over leadership (replica promote, CLUSTER FAILOVER, or SENTINEL FAILOVER) to an already updated replica. The old primary restarts as a regular replica, cutting downtime virtually to zero. On Valkey 9.0+ images, the shutdown-on-sigterm failover directive is also enabled.
Managing the cluster is convenient via built-in toggles and the CLI extension:
valkey.wellcake.io/proactive-rollout: "true"— zero-downtime updates with proactive primary role handover;valkey.wellcake.io/rotate-password— on-the-fly password rotation viaACL SETUSERwithout pod restarts;kubectl valkeyplugin — quick status checks (status), backups (backup), resharding (reshard), primary failover (failover), and cluster pausing (hibernate).
For security, replicas connect to the primary under a dedicated replicator account (+psync +replconf +ping) with no access to keys. Updated TLS certificates are picked up on the fly via CONFIG SET tls-cert-file (plus automatic checks with tls-auto-reload-interval 3600). S3 backups run on a CronJob schedule via valkey-cli --rdb with restoration from spec.restoreFrom, while spec.replicateFrom sets up asynchronous cross-region replication with certificate bundling (replicateFrom.caSecret) for mutual TLS.
Where wellcake Saves Team Resources and When to Hold Off
Controllers like wellcake show how automation is moving out of cloud consoles into the open ecosystem (we discussed the growth of such solutions in our review of six Kubernetes platform engineering tools). Still, every tool has its operating conditions and limits.
The operator is a great fit for teams running their own Kubernetes clusters on bare-metal or across multiple clouds that want to manage a Valkey fleet declaratively via GitOps, separate read and write traffic across services, and automate resharding alongside S3 snapshots.
You should avoid adopting it in two cases:
- Mini-projects and disposable environments. For a small service where losing the cache on restart goes unnoticed, installing an operator and CRDs is unnecessary complexity; a single container is enough.
- Mission-critical financial systems with strict SLAs. The authors explicitly mark the project status as
early / experimental: APIs are atv1alpha1andv1beta1, and validation webhooks are disabled by default (--set webhooks.enabled=true). You should battle-test the operator in a staging environment before running it in production.
Moving to Valkey in Kubernetes no longer forces you to write fragile failover scripts or tolerate write pauses during upgrades: wellcake packs four topologies, seamless failover, and S3 backups into a single resource. If you want to explore the controller source code, read the notes in docs/adr/, or try the kubectl valkey plugin, check out the documentation and example manifests in the project repository.

