Part of the series: From Zero to Kubernetes Operators with Kubebuilder (KubeAgent Journey)
→ Full code for this post: github.com/madmmas/kubeoperator-journey/tree/blog-03 → Generated scaffold reference: kubebuilder/ directory in the repo root
In Blog 2 we built a working Kubernetes-style control loop from scratch — a store, a watch loop, a work queue, and a Reconcile() function. Every component was explicit and observable.
In this post we let Kubebuilder generate all of that infrastructure. Then we read every generated file and map it back to the components we built manually. Nothing will be surprising. Everything will be familiar.
By the end you will have:
- A working Kubebuilder project scaffold
- A
DatabaseClusterCRD with production-ready types - A complete mental map of what Kubebuilder owns vs what you own
Prerequisites
You need three tools. Install them once, use them for the entire series.
# 1. kind — local Kubernetes cluster (no cloud account needed)
go install sigs.k8s.io/kind@v0.22.0
# 2. kubectl — Kubernetes CLI
# macOS
brew install kubectl
# Linux
curl -LO "https://dl.k8s.io/release/$(curl -Ls https://dl.k8s.io/release/stable.txt)/bin/linux/amd64/kubectl"
chmod +x kubectl && sudo mv kubectl /usr/local/bin/
# 3. Kubebuilder — the scaffolding tool
curl -L -o kubebuilder \
https://github.com/kubernetes-sigs/kubebuilder/releases/download/v3.14.0/kubebuilder_linux_amd64
chmod +x kubebuilder && sudo mv kubebuilder /usr/local/bin/
# Verify
kind version # v0.22.0
kubectl version # Client Version: v1.29.x
kubebuilder version # KubeBuilderVersion:"3.14.0"
Create your local cluster:
kind create cluster --name kubeoperator-dev
kubectl cluster-info --context kind-kubeoperator-dev
Initialising the Project
Navigate to your kubeoperator-journey repo and run:
kubebuilder init \
--domain madmmas.dev \
--repo github.com/madmmas/kubeoperator-journey
Two flags matter here:
--domain madmmas.dev sets the API group domain. Your CRD's apiVersion will be databases.madmmas.dev/v1alpha1. Pick your own domain — anything you control works.
--repo github.com/madmmas/kubeoperator-journey sets the Go module path. It must match your go.mod exactly.
Kubebuilder generates this project structure:
kubeoperator-journey/
├── cmd/
│ └── main.go ← operator entry point
├── config/
│ ├── default/ ← Kustomize overlay (production deployment)
│ ├── manager/ ← Deployment manifest for the operator pod
│ ├── rbac/ ← ServiceAccount, ClusterRole, bindings
│ └── prometheus/ ← Metrics scrape config
├── hack/
│ └── boilerplate.go.txt ← License header template
├── test/
│ └── e2e/ ← End-to-end test skeleton
├── Dockerfile ← Multi-stage build for the operator image
├── Makefile ← All the commands you need
├── PROJECT ← Kubebuilder project metadata
└── go.mod
Now generate the API and controller for our DatabaseCluster resource:
kubebuilder create api \
--group databases \
--version v1alpha1 \
--kind DatabaseCluster \
--resource --controller
This adds:
├── api/
│ └── v1alpha1/
│ ├── databasecluster_types.go ← YOU EDIT THIS
│ └── groupversion_info.go ← Kubebuilder owns this
└── internal/
└── controller/
├── databasecluster_controller.go ← YOU EDIT THIS
└── databasecluster_controller_test.go
Two files are yours. Everything else is infrastructure.
File by File: What Kubebuilder Generated
PROJECT — The Manifest of Intent
domain: madmmas.dev
layout:
- go.kubebuilder.io/v4
projectName: kubeoperator-journey
repo: github.com/madmmas/kubeoperator-journey
resources:
- api:
crdVersion: v1
namespaced: true
controller: true
group: databases
kind: DatabaseCluster
version: v1alpha1
version: "3"
Kubebuilder reads this file before every create api or plugin command. It's how Kubebuilder knows the context of your project — your domain, your Go module, every API you've registered. Don't hand-edit it; Kubebuilder updates it automatically.
cmd/main.go — The Manager: Our Blog 2 Controller Loop, Production-Grade
This is the operator's main(). Map it to what we built in Blog 2:
mgr, err := ctrl.NewManager(ctrl.GetConfigOrDie(), ctrl.Options{
Scheme: scheme,
Metrics: metricsserver.Options{
BindAddress: metricsAddr,
},
HealthProbeBindAddress: probeAddr,
LeaderElection: enableLeaderElection,
LeaderElectionID: "ae19e053.madmmas.dev",
})
ctrl.NewManager is our Blog 2 Controller — but production-hardened. It manages:
- The watch loop (Informers for every registered resource type)
- The work queue (rate-limited, with exponential backoff)
- The reconcile goroutine pool (configurable concurrency)
- Leader election (ensures only one replica of your operator runs at a time in HA deployments)
- Health endpoints (
/healthzand/readyzfor Kubernetes liveness/readiness probes) - Metrics (Prometheus endpoint at
:8080)
In Blog 2 we started two goroutines manually. Manager starts dozens of goroutines and handles their entire lifecycle. You call mgr.Start() and it runs until the process receives a shutdown signal.
// After we add our reconciler (shown below), this is all that's needed:
if err := mgr.Start(ctrl.SetupSignalHandler()); err != nil {
setupLog.Error(err, "problem running manager")
os.Exit(1)
}
The LeaderElectionID is a lock resource name in your cluster. Two operator pods race to acquire this lock; only one becomes leader and runs reconciliation. The other standby pods watch the lock and take over if the leader dies. This is how you run operators with zero downtime.
api/v1alpha1/groupversion_info.go — The Scheme Registration
var (
GroupVersion = schema.GroupVersion{
Group: "databases.madmmas.dev",
Version: "v1alpha1",
}
SchemeBuilder = &scheme.Builder{GroupVersion: GroupVersion}
AddToScheme = SchemeBuilder.AddToScheme
)
Kubernetes uses a scheme to map Go types to API group/version/kind strings. This file registers DatabaseCluster into that scheme so that when the kube-apiserver sends a JSON blob with apiVersion: databases.madmmas.dev/v1alpha1, kind: DatabaseCluster, the controller-runtime library knows which Go struct to deserialise it into.
You don't touch this file. Kubebuilder updates it when you add more APIs.
api/v1alpha1/databasecluster_types.go — The CRD: Your Domain Model
This is the most important file Kubebuilder generates, and the one you'll spend the most time in. The scaffold gives you a skeleton:
// DatabaseClusterSpec defines the desired state of DatabaseCluster
type DatabaseClusterSpec struct {
// Foo is an example field of DatabaseCluster. Edit databasecluster_types.go to remove/update
Foo string `json:"foo,omitempty"`
}
// DatabaseClusterStatus defines the observed state of DatabaseCluster
type DatabaseClusterStatus struct {
// INSERT ADDITIONAL STATUS FIELD - define observed state of cluster
}
//+kubebuilder:object:root=true
//+kubebuilder:subresource:status
type DatabaseCluster struct {
metav1.TypeMeta `json:",inline"`
metav1.ObjectMeta `json:"metadata,omitempty"`
Spec DatabaseClusterSpec `json:"spec,omitempty"`
Status DatabaseClusterStatus `json:"status,omitempty"`
}
The Foo string field is a placeholder — delete it and replace with your real domain model. Here's what our DatabaseCluster types look like after filling them in (the full version is at kubebuilder/api/v1alpha1/databasecluster_types.go):
type DatabaseClusterSpec struct {
// +kubebuilder:validation:Minimum=1
// +kubebuilder:validation:Maximum=10
Replicas int32 `json:"replicas"`
// +kubebuilder:validation:Pattern=`^\d+\.\d+$`
Version string `json:"version"`
// +optional
StorageSize string `json:"storageSize,omitempty"`
// +optional
BackupSchedule string `json:"backupSchedule,omitempty"`
}
type DatabaseClusterStatus struct {
Phase DatabaseClusterPhase `json:"phase,omitempty"`
ReadyReplicas int32 `json:"readyReplicas,omitempty"`
CurrentVersion string `json:"currentVersion,omitempty"`
LastBackupTime *metav1.Time `json:"lastBackupTime,omitempty"`
Conditions []metav1.Condition `json:"conditions,omitempty"`
}
Four concepts worth understanding deeply here:
1. Spec vs Status — the contract boundary
Spec is yours to write. Status belongs to the operator. Never put desired state in Status, never put observed state in Spec. This boundary is enforced by the //+kubebuilder:subresource:status marker — it creates a separate API endpoint for status updates, meaning your controller can update Status without accidentally triggering a resourceVersion conflict with users updating Spec.
2. Marker comments are code
The //+kubebuilder:validation:Minimum=1 lines above look like comments but they're not — they're processed by controller-gen (run via make generate) to produce validation rules in the generated CRD YAML:
spec:
replicas:
type: integer
minimum: 1
maximum: 10
version:
type: string
pattern: ^\d+\.\d+$
These validations run in the kube-apiserver before your controller ever sees the resource. A user trying to apply replicas: 0 gets an immediate rejection — your operator doesn't need to handle that case defensively.
3. The Conditions pattern
Conditions []metav1.Condition `json:"conditions,omitempty"`
metav1.Condition is the Kubernetes standard for expressing resource state. Each condition has a Type (e.g. "Ready"), Status ("True" / "False" / "Unknown"), Reason (machine-readable), and Message (human-readable). This is what kubectl describe shows and what monitoring tools like Prometheus alert on.
Always use standard conditions over custom status fields where possible — tooling across the ecosystem knows how to read them.
4. printcolumn markers
// +kubebuilder:printcolumn:name="Phase",type=string,JSONPath=`.status.phase`
// +kubebuilder:printcolumn:name="Ready",type=string,JSONPath=`.status.readyReplicas`
// +kubebuilder:printcolumn:name="Version",type=string,JSONPath=`.status.currentVersion`
// +kubebuilder:printcolumn:name="Age",type=date,JSONPath=`.metadata.creationTimestamp`
These control kubectl get databasecluster output:
NAME PHASE READY VERSION AGE
production-postgres Running 3 15.4 2d
staging-postgres Running 1 15.4 1d
Without these, kubectl get shows only NAME and AGE. Add columns for whatever operators and users need to see at a glance.
internal/controller/databasecluster_controller.go — The Reconciler Skeleton
type DatabaseClusterReconciler struct {
client.Client
Scheme *runtime.Scheme
}
//+kubebuilder:rbac:groups=databases.madmmas.dev,resources=databaseclusters,verbs=get;list;watch;create;update;patch;delete
//+kubebuilder:rbac:groups=databases.madmmas.dev,resources=databaseclusters/status,verbs=get;update;patch
//+kubebuilder:rbac:groups=databases.madmmas.dev,resources=databaseclusters/finalizers,verbs=update
func (r *DatabaseClusterReconciler) Reconcile(ctx context.Context, req ctrl.Request) (ctrl.Result, error) {
_ = log.FromContext(ctx)
// TODO(user): your logic here
return ctrl.Result{}, nil
}
func (r *DatabaseClusterReconciler) SetupWithManager(mgr ctrl.Manager) error {
return ctrl.NewControllerManagedBy(mgr).
For(&databasesv1alpha1.DatabaseCluster{}).
Complete(r)
}
There are three pieces here worth mapping back to Blog 2:
DatabaseClusterReconciler struct — This embeds client.Client, which is the production version of our Blog 2 store.Get(). Instead of an in-memory map, it talks to the kube-apiserver over HTTPS. r.Get(), r.List(), r.Create(), r.Update(), r.Delete() — all the operations you need to interact with the cluster.
The RBAC markers — These //+kubebuilder:rbac:... lines generate the config/rbac/role.yaml ClusterRole. make manifests reads these markers and writes the YAML. This means your operator's permissions are defined next to the code that needs them, not in a separate file that drifts out of sync.
SetupWithManager — This is where our Blog 2 watchLoop and reconcileLoop get connected to the Manager. For(&DatabaseCluster{}) tells the Manager: "watch this resource type and call my Reconcile() when anything changes." One line replaces our entire two-goroutine setup.
Reconcile(ctx, req) — The function signature maps exactly to Blog 2:
| Blog 2 | controller-runtime |
|---|---|
key string | req ctrl.Request (contains NamespacedName) |
ReconcileResult{} | ctrl.Result{} |
ReconcileResult{Requeue: true, RequeueAfter: d} | ctrl.Result{RequeueAfter: d} |
return result, err where err != nil → backoff requeue | return ctrl.Result{}, err |
Makefile — Your Daily Driver
You won't memorise all the kubebuilder and controller-gen commands. The Makefile does it for you. The ones you'll use constantly:
make generate # Regenerate DeepCopy methods after editing types
make manifests # Regenerate CRD YAML and RBAC from marker comments
make install # Apply CRDs to your local kind cluster
make run # Run the operator locally (connects to kind cluster)
make test # Run unit + integration tests via envtest
A common mistake: editing api/v1alpha1/databasecluster_types.go and running make install without make generate first. The CRD YAML won't reflect your changes. The correct sequence is always:
make generate # 1. regenerate Go code (DeepCopy methods)
make manifests # 2. regenerate CRD YAML from markers
make install # 3. apply updated CRDs to the cluster
Or just run make (the default all target), which runs all three in the right order.
config/ — The Deployment Package
The config/ directory is a Kustomize application that describes how to deploy your operator to a real cluster:
config/
├── default/ ← Kustomize overlay; composes everything below
├── manager/ ← Deployment (your operator pod)
│ └── manager.yaml
├── rbac/ ← ServiceAccount, ClusterRole, RoleBinding
│ ├── role.yaml ← Generated from your //+kubebuilder:rbac markers
│ └── ...
└── prometheus/ ← ServiceMonitor for scraping /metrics
The manager.yaml Deployment has production settings baked in:
securityContext:
runAsNonRoot: true
containers:
- command:
- /manager
args:
- --leader-elect
livenessProbe:
httpGet:
path: /healthz
port: 8081
initialDelaySeconds: 15
readinessProbe:
httpGet:
path: /readyz
port: 8081
resources:
limits:
cpu: 500m
memory: 128Mi
Leader election enabled. Liveness and readiness probes configured. Non-root security context. Resource limits set. This is a production-quality deployment configuration from the first scaffold — not something you'd write by hand and later realise was missing security constraints.
The Full Picture: What You Own vs What Kubebuilder Owns
After running two commands (kubebuilder init and kubebuilder create api), here's the ownership map:
┌─────────────────────────────────────────────────────────────┐
│ KUBEBUILDER OWNS (don't hand-edit) │
│ │
│ api/v1alpha1/groupversion_info.go scheme registration │
│ config/ deployment manifests │
│ Dockerfile container build │
│ Makefile build commands │
│ PROJECT project metadata │
│ hack/boilerplate.go.txt license header │
│ │
├─────────────────────────────────────────────────────────────┤
│ YOU OWN (this is where operator logic lives) │
│ │
│ api/v1alpha1/databasecluster_types.go ← domain model │
│ internal/controller/*_controller.go ← Reconcile() │
│ │
└─────────────────────────────────────────────────────────────┘
Two files. That's the surface area of your operator. Everything else is generated infrastructure that you benefit from without maintaining.
Connecting to Blog 2: The Same Loop, Different Packaging
The Blog 2 components map cleanly to what Kubebuilder generates:
| Blog 2 component | Kubebuilder equivalent |
|---|---|
watcher.NewStore() | kube-apiserver + etcd (external) |
store.Watch() | Informer.AddEventHandler() inside Manager |
watchLoop() goroutine | Manager's internal Informer goroutines |
workQueue chan string | workqueue.RateLimitingInterface |
reconcileLoop() goroutine | Manager's worker goroutine pool |
Reconcile(key string) | Reconcile(ctx, req ctrl.Request) |
store.Get(key) | r.Get(ctx, req.NamespacedName, &obj) |
ReconcileResult{Requeue: true} | ctrl.Result{RequeueAfter: d} |
ctrl.Stop() | ctrl.SetupSignalHandler() |
The pattern is identical. The implementation is production-grade.
Applying Your CRD to the Local Cluster
With your types defined and kind running:
make generate # generate DeepCopy methods
make manifests # generate CRD YAML from marker comments
make install # apply CRDs to kind cluster
# Verify the CRD is registered
kubectl get crds | grep databasecluster
# databaseclusters.databases.madmmas.dev 2026-06-30T09:00:00Z
# Verify the API is available
kubectl api-resources | grep databases
# databaseclusters databases.madmmas.dev/v1alpha1 true DatabaseCluster
# Create an instance
kubectl apply -f - <<EOF
apiVersion: databases.madmmas.dev/v1alpha1
kind: DatabaseCluster
metadata:
name: production-postgres
namespace: default
spec:
replicas: 3
version: "15.4"
storageSize: "100Gi"
backupSchedule: "0 2 * * *"
EOF
# See our printcolumn markers in action
kubectl get databasecluster
# NAME PHASE READY VERSION AGE
# production-postgres <none> 0 <none> 5s
Phase and Ready are empty because we haven't written Reconcile() yet — the controller isn't running. But the resource exists in the cluster, the API is live, and the validation rules we defined in our markers are already enforcing the schema.
Try applying an invalid resource:
kubectl apply -f - <<EOF
apiVersion: databases.madmmas.dev/v1alpha1
kind: DatabaseCluster
metadata:
name: invalid-cluster
namespace: default
spec:
replicas: 0 # violates minimum=1
version: "15.4"
EOF
# The apiserver rejects it immediately:
# The DatabaseCluster "invalid-cluster" is invalid:
# spec.replicas: Invalid value: 0: spec.replicas in body should be >= 1
Validation at the API layer — before your controller ever runs — is one of the most valuable things CRD markers give you.
What's Next
In Blog 4 we write the Reconcile() function. The scaffold gives us an empty function body with a TODO comment. We fill it in: fetch the DatabaseCluster from the API, provision a StatefulSet to match its spec, update the status conditions, and handle the not-found case for deletions.
For the first time in this series, the operator will actually do something when you apply a DatabaseCluster resource.
This is Blog 3 of the series "From Zero to Kubernetes Operators with Kubebuilder (KubeAgent Journey)." The series covers Kubernetes foundations, real operator development with Kubebuilder, and evolving into intelligent KubeAgent-style systems.