Temporal Worker Controller

🚀 Generally Available: This project is Generally Available and ready for production use cases. Core functionality is complete with stable APIs.

The Temporal Worker Controller makes it simple and safe to deploy Temporal workers on Kubernetes.

Temporal workflows require deterministic execution, which means updating worker code can break running workflows if the changes aren't backward compatible. Traditional deployment strategies force you to either risk breaking existing workflows or use Temporal's Patching API to maintain compatibility across versions.

Temporal's Worker Versioning feature solves this dilemma by providing programmatic control over worker versions and traffic routing. The Temporal Worker Controller automates a deployment system that uses Worker Versioning on Kubernetes. When you deploy new code, the controller automatically creates a new worker version while keeping the old version running. Existing workflows continue on the old version while new workflows use the new version. This approach eliminates the need for patches in many cases and ensures running workflows are never disrupted.

What does it do?

🔒 Protected Pinned workflows - Workflows pinned to a version stay on that version and won't break
🎚️ Controlled rollout for AutoUpgrade workflows - AutoUpgrade workflows shifted to new versions with configurable safety controls
📦 Automatic version management - Registers versions with Temporal, manages routing rules, and tracks version lifecycle
🎯 Smart traffic routing - New workflows automatically get routed to your target worker version
🛡️ Progressive rollouts - Catch incompatible changes early with small traffic percentages before they spread
⚡ Easy rollbacks - Instantly route traffic back to a previous version if issues are detected
📈 Per-version autoscaling - Attach HPAs or other custom scalers to each versioned Deployment via WorkerResourceTemplate

Quick Example

Instead of this traditional approach where deployments can break running workflows:

# ❌ Traditional deployment - risky for running workflows
apiVersion: apps/v1
kind: Deployment
metadata:
  name: my-worker
spec:
  template:
    spec:
      containers:
      - name: worker
        image: my-worker:v2.0.0  # This change might break existing workflows!

You define your worker like this:

# ✅ Temporal Worker Controller - safe deployments
apiVersion: temporal.io/v1alpha1
kind: WorkerDeployment
metadata:
  name: my-worker
spec:
  deployment:
    replicas: 3
    template:
      spec:
        containers:
          - name: worker
            image: my-worker:v2.0.0  # Safe to deploy!
  rollout:
    strategy: Progressive  # Gradual, safe rollout
    steps:
      - rampPercentage: 10
        pauseDuration: 5m
      - rampPercentage: 50
        pauseDuration: 10m

When you update the image, the controller automatically:

1. 🆕 Creates a new deployment with your updated worker
2. 📊 Gradually routes new workflows and AutoUpgrade workflows to the new version
3. 🔒 Keeps Pinned workflows running on their original version (guaranteed safety)
4. 🧹 Automatically scales down and cleans up old versions once they are drained

🏃‍♂️ Getting Started

Prerequisites

• Kubernetes cluster (1.19+)

• Helm v3.0+ if deploying via our Helm chart

• Temporal Server (Cloud or self-hosted v1.29.1)

• Basic familiarity with Temporal Workers, Workflows, and Worker Versioning

• TLS for the validating webhook (required for WorkerResourceTemplate) — the recommended path is cert-manager, which handles certificate provisioning automatically. Install it separately before installing the controller chart. If you prefer to manage TLS yourself, see Webhook TLS.

🔧 Installation

CRDs are shipped as a separate Helm chart so they can be upgraded independently of the controller. Install the CRDs chart first, then the controller chart:

# 1. Install CRDs
helm install temporal-worker-controller-crds \
  oci://docker.io/temporalio/temporal-worker-controller-crds \
  --version <version> \
  --namespace <your-namespace> \
  --create-namespace

# 2. Install the controller
helm install temporal-worker-controller \
  oci://docker.io/temporalio/temporal-worker-controller \
  --version <version> \
  --namespace <your-namespace>

See docs/crd-management.md for upgrade, rollback, and migration instructions.

Next Steps

New to deploying workers with this controller? → Start with our Migration Guide to learn how to safely transition from traditional deployments.

Setting up CI/CD for steady-state rollouts? → See the CD Rollouts Guide for Helm, kubectl, ArgoCD, and Flux integration patterns.

Ready to dive deeper? → Check out the Architecture Guide to understand how the controller works, or the Temporal Worker Versioning docs to learn about the underlying Temporal feature.

Need configuration help? → See the Configuration Reference for all available options.

Webhook TLS Configuration

The validating webhook requires TLS. Choose one of the following options.

Option 1: cert-manager (default)

With certmanager.enabled: true (the default), the chart creates an Issuer and Certificate resource. cert-manager generates the TLS certificate and stores it in a Secret. Cert-manager must be installed independently in the cluster before installing TWC.

# Install cert-manager (once per cluster)
helm install cert-manager jetstack/cert-manager \
  --namespace cert-manager \
  --create-namespace \
  --set crds.enabled=true

# Install TWC (cert-manager creates the webhook cert automatically)
helm install temporal-worker-controller <chart> \
  --namespace temporal-system \
  --set certmanager.enabled=true

Option 2: Bring your own certificate

If you manage TLS certificates outside of cert-manager, create the Secret yourself and tell TWC its name:

# Create your TLS Secret
kubectl create secret tls my-webhook-cert \
  --cert=webhook.pem \
  --key=webhook-key.pem \
  --namespace temporal-system

# Install TWC pointing at your Secret
helm install temporal-worker-controller <chart> \
  --namespace temporal-system \
  --set certmanager.enabled=false \
  --set webhook.certSecretName=my-webhook-cert \
  --set certmanager.caBundle=$(base64 < ca.pem)

The webhook.certSecretName value (default: webhook-server-cert) controls which Secret the controller pod mounts for TLS. The caBundle value tells the Kubernetes API server which CA to trust when calling the webhook.

Option 3: Self-managed with cert-manager

If you use cert-manager but want to control the Secret name:

helm install temporal-worker-controller <chart> \
  --namespace temporal-system \
  --set certmanager.enabled=true \
  --set webhook.certSecretName=my-custom-cert-name

Features

• ✅ Registration of new Temporal Worker Deployment Versions
• ✅ Creation of versioned Deployment resources (managing Pods that run your Temporal workers)
• ✅ Automatic lifecycle scaling - Scales down worker versions when no longer needed
• ✅ Deletion of resources associated with drained Worker Deployment Versions
• ✅ Multiple rollout strategies: Manual, AllAtOnce, and Progressive rollouts
• ✅ Gate workflows - Test new versions with a pre-deployment test before routing real traffic to them
• ✅ Per-version attached resources - Attach HPAs, PodDisruptionBudgets, or any namespaced Kubernetes resource to each worker version with running workers via WorkerResourceTemplate — this is also the recommended path for metric-based and backlog-based autoscaling

💡 Why Use This?

Manual Worker Versioning is Complex

While Temporal's Worker Versioning feature solves deployment safety problems, using it manually requires:

• Manual API calls - Register versions, manage routing rules, track version states
• Infrastructure coordination - Deploy multiple Kubernetes resources for each version
• Lifecycle monitoring - Watch for drained versions and clean up resources
• Rollout orchestration - Manually control progressive traffic shifting

The Controller Automates Everything

The Temporal Worker Controller eliminates this operational overhead by automating the entire Worker Versioning lifecycle on Kubernetes:

• Automatic Temporal integration - Registers versions and manages routing without manual API calls
• Kubernetes-native workflow - Update a single custom resource, get full rainbow deployments
• Intelligent cleanup - Monitors version drainage and automatically removes unused resources
• Built-in rollout strategies - Progressive, AllAtOnce, and Manual with configurable safety controls

📖 Documentation

Document	Description
Releases	How we version and release the controller and Helm Chart
Migration Guide	Step-by-step guide for migrating from traditional deployments
Reversion Guide	Step-by-step guide for migrating back to unversioned deployment
CD Rollouts	Helm, kubectl, ArgoCD, and Flux integration for steady-state rollouts
Architecture	Technical deep-dive into how the controller works
Configuration	Complete configuration reference
Concepts	Key concepts and terminology
Limits	Technical constraints and limitations
WorkerResourceTemplate	Attach HPAs, PDBs, and other resources to each versioned Deployment
CRD Management	CRD upgrade, rollback, and migration guide

🔧 Worker Configuration

Your workers need these environment variables (automatically set by the controller):

TEMPORAL_ADDRESS=your-temporal-server:7233
TEMPORAL_NAMESPACE=your-namespace  
TEMPORAL_DEPLOYMENT_NAME=my-worker        # Unique worker deployment name
TEMPORAL_WORKER_BUILD_ID=v1.2.3          # Version identifier

Important: Don't set the above environment variables manually - the controller manages these automatically.

🤝 Contributing

We welcome all contributions! This includes:

• 🔧 Code contributions - Please start by opening an issue to discuss your idea
• 🐛 Bug reports - File an issue
• 💡 Feature requests - Tell us what you'd like to see
• 💬 Feedback - Join #safe-deploys on Temporal Slack

🛠️ Development

Want to try the controller locally? Check out the local demo guide for development setup.

Need a test controller image from an unmerged branch? Run the publish-branch-image GitHub Actions workflow with the branch name.

📄 License

This project is licensed under the MIT License.

---

Questions? Reach out to @jlegrone or the #safe-deploys channel on Temporal Slack!