Kubernetes Deployment Checklist: From Manifest Validation to Production Rollout

Overview

Kubernetes makes it possible to describe application workloads declaratively, but a valid manifest is only the starting point. A Deployment can be accepted by the API server and still fail because an image cannot be pulled, a Secret is missing, a probe is too strict, a Service selects the wrong labels, or the workload has no realistic resource allocation.

Use this Kubernetes deployment checklist as a release gate rather than a one-time document. Run the checks in a non-production environment first, record the expected behavior, and make the final production decision using evidence from CI/CD, cluster events, application logs, and service-level monitoring. The exact commands and available fields can vary by Kubernetes release and installed extensions, so verify unfamiliar fields against the target cluster.

For teams building a broader delivery process, this guide works well alongside a CI/CD pipeline design guide and a curated DevOps tools directory. The aim is not to add ceremony to every change, but to make high-risk assumptions visible before they affect users.

Checklist by scenario

1. Before applying a new or changed manifest

• Confirm the target: Check the cluster context, namespace, account, and intended environment before running any apply or delete command.
• Pin the image: Prefer an immutable tag or digest that can be traced to a source revision. Confirm the image exists in the expected registry and that nodes can authenticate to it.
• Review the rendered configuration: Inspect the final output after Helm, Kustomize, or another templating system has processed values. Do not review only the template when environment-specific substitutions can change behavior.
• Validate the object: Use client-side or server-side dry runs where supported, and compare the result with the API resources available on the target cluster.
• Check ownership and scope: Confirm namespace, labels, annotations, owner references, and selectors. A selector that does not match Pod labels can leave a Service or Deployment disconnected from its workload.
• Check dependencies: Identify required ConfigMaps, Secrets, service accounts, persistent volumes, admission policies, and external services. Decide whether each dependency must exist before rollout.

A small, repeatable manifest check can be included in a pipeline. Adjust file paths and tools to your repository:

kubectl config current-context
kubectl get namespace <namespace>
kubectl apply --dry-run=server -f rendered-manifest.yaml
kubectl diff -f rendered-manifest.yaml
kubectl explain deployment.spec.template.spec.containers --recursive

Use kubectl explain against the target API server when you need to confirm field names or nesting. This is especially useful when a manifest was written for a different Kubernetes version or relies on a feature that may not be enabled in every cluster.

2. When preparing the container and application

• Make startup predictable: The container should exit clearly on unrecoverable initialization errors rather than remaining alive but unusable.
• Handle termination: Ensure the application responds to termination signals, stops accepting new work when appropriate, and finishes within the configured termination period.
• Separate configuration from the image: Keep environment-specific values outside the image, and verify that required variables have safe validation and useful error messages.
• Expose the correct port: Confirm the process listens on the expected interface and port. A declared container port does not itself make an application reachable.
• Test health endpoints: Make readiness represent the ability to receive traffic, liveness represent a condition requiring restart, and startup checks protect applications with slow initialization.
• Check compatibility: Verify the image architecture, runtime libraries, filesystem expectations, and permissions against the nodes that may schedule the Pod.

Probe paths and timeouts should be tested under realistic startup and dependency conditions. A liveness probe that runs before the application can initialize may create a restart loop; a readiness probe that never reflects dependency failure can send traffic to a degraded instance.

3. When reviewing configuration, secrets, and identity

• Confirm every referenced ConfigMap key and Secret key exists in the target namespace.
• Do not place credentials, private keys, or tokens directly in a committed manifest. Treat encoded values as recoverable data, not encryption.
• Use the narrowest service account permissions the workload needs. Review mounted tokens and remove automatic access where the application does not use the Kubernetes API.
• Check that configuration changes have an intentional restart or reload behavior. A changed ConfigMap does not guarantee that an application reloads its process state.
• Verify external identity, registry access, and cloud permissions from the actual Pod service account or workload identity, not only from an administrator workstation.

For related infrastructure and governance work, review your network boundaries and policy controls with the same deployment context. The guide to policy as code tools can help teams place repeatable checks around platform changes.

4. When setting resources and scheduling

• Set requests: CPU and memory requests help the scheduler make placement decisions. They should describe expected baseline consumption, not an arbitrary default.
• Set limits deliberately: Consider the effect of CPU throttling, memory termination, and workload bursts before choosing limits.
• Check capacity: Confirm the cluster has room for the desired replica count plus surge capacity during a rolling update.
• Review placement rules: Taints, tolerations, affinity, topology spread, node selectors, and architecture constraints can all prevent scheduling or concentrate replicas on one failure domain.
• Plan disruption: If the workload needs availability during maintenance, define an appropriate disruption budget and test it against the number of healthy replicas.

5. When securing the workload

• Run as a non-root user where the application supports it.
• Set an appropriate filesystem policy, such as read-only root storage, and provide writable volumes only where required.
• Drop unnecessary Linux capabilities and avoid privileged containers unless there is a documented platform requirement.
• Review host namespace access, host path mounts, and device access carefully.
• Apply network policies where the cluster model supports them, beginning with documented ingress and egress requirements rather than assuming all traffic is needed.
• Scan the image and dependencies through the team’s normal security process, then record exceptions with an owner and review date.

6. When exposing and rolling out the application

• Confirm Service selectors match the intended Pod labels and that the Service port maps to the container’s listening port.
• Review Ingress or Gateway routes, TLS references, hostnames, timeouts, and authentication behavior.
• Choose a rollout strategy that matches the failure cost. A standard rolling update may be sufficient for a low-risk stateless change; a canary or staged approach may be better when traffic or compatibility risk is high.
• Set rollout parameters such as maximum unavailable and maximum surge so capacity and availability expectations are explicit.
• Define the observation window and success signals before starting. A rollout is not complete merely because all Pods report running.

Keep rollback practical. A previous image reference, configuration version, and migration plan should be identifiable. If a release changes a database schema, confirm whether the old application version can still operate before relying on an application rollback.

7. During and after the rollout

• Watch Deployment and ReplicaSet conditions, Pod readiness, restart counts, events, and scheduling status.
• Verify that endpoints or endpoint slices contain the expected ready backends.
• Run a focused smoke test through the same Service, Ingress, or Gateway path that users will use.
• Check application error rates, latency, saturation, queue depth, and dependency health.
• Confirm logs and metrics include enough workload, namespace, version, and request context to distinguish the new release.
• Record the deployed image, manifest revision, configuration version, and operator or pipeline result.

For alert design, pair deployment verification with an SRE alert fatigue checklist. Alerts should help identify a real regression without turning every routine restart or rollout event into an incident.

What to double-check

Version and API compatibility

Kubernetes APIs, admission behavior, ingress implementations, storage drivers, and policy engines can differ between clusters. Before promotion, inspect the target server’s supported resources and validate the rendered manifest there. Avoid assuming that a manifest accepted in a development cluster will behave identically in production.

Pay particular attention to deprecated or removed API versions, fields supplied by custom resources, and settings controlled by admission policies. Use the cluster’s documentation and discovery output as the authority for the environment you are deploying to.

Data and migration safety

Application rollout and data migration should be considered together. Check backup or recovery expectations, migration ordering, lock duration, backward compatibility, and the behavior of a partially completed migration. A successful Pod rollout does not prove that the data change is safe.

Capacity during failure

Ask what happens when one replica, one node, or one dependency is unavailable. Confirm that requests, disruption budgets, topology rules, and autoscaling settings do not conflict. Also check whether a rollback needs more capacity than the forward rollout.

Operational ownership

Every production deployment should have a clear owner, a way to identify the release, and a documented rollback or escalation path. Internal developer portals and platform engineering tools can make these details visible at deployment time; see the comparison of internal developer portal tools for broader platform workflow considerations.

Common mistakes

• Using mutable image tags: The same tag can point to different content, making reproduction and rollback harder. Use traceable image references.
• Confusing running with ready: A running container may still be failing its readiness check or returning application errors.
• Skipping server-side validation: Local tooling may not know the target cluster’s API versions, admission rules, or custom resources.
• Omitting resource requests: Without realistic requests, scheduling and capacity decisions become less predictable.
• Making liveness do too much: A temporary dependency failure can cause unnecessary restarts when liveness is used as a general health check.
• Testing rollback only in theory: A rollback command is not a rollback plan unless the prior image, configuration, schema compatibility, and verification steps are known.
• Ignoring labels and selectors: A small label mismatch can leave a Service without endpoints or cause a controller to manage the wrong Pods.
• Deploying without observing the user path: Direct Pod checks can pass while DNS, TLS, routing, authentication, or the Service path is broken.
• Embedding secrets in repositories: Base64 encoding is not a substitute for secret protection, rotation, access control, or external secret management.

When a deployment fails, use this short decision tree:

1. Is the Pod pending? Check scheduling events, resource requests, node selectors, taints, affinity, and available capacity.
2. Is the Pod stuck pulling an image? Check the image reference, registry reachability, image-pull credentials, architecture, and node events.
3. Is the container restarting? Inspect logs from the current and previous container, then review command, arguments, environment, mounted files, permissions, and liveness behavior.
4. Is the Pod running but not ready? Test the readiness or startup endpoint from inside the relevant network path and inspect dependency failures.
5. Are Pods ready but traffic failing? Check Service selectors, endpoints, ports, routing rules, TLS, network policies, and application logs.
6. Did the rollout stall? Inspect Deployment conditions, unavailable replicas, readiness failures, disruption constraints, and capacity for surge replicas. Pause or roll back according to the pre-agreed release plan.

When to revisit

Revisit this Kubernetes production checklist before each material release, not only when application code changes. It should be part of seasonal planning cycles, platform upgrades, cluster migrations, and changes to the delivery workflow. Update it when the Kubernetes version, admission controls, ingress or Gateway implementation, storage class, service mesh, cloud identity model, or observability stack changes.

Also review the checklist after an incident, a failed rollback, a security finding, or a capacity surprise. Add the missing check to the appropriate pipeline or deployment template rather than relying solely on individual memory. If your workflow includes scheduled jobs or maintenance tasks, validate their schedules with a cron expression builder and document the expected operational impact.

For the next deployment, copy the rendered manifests, identify the target context, run server-side validation, confirm image and dependency readiness, check resources and security, define success signals, and write down the rollback trigger before applying the change. After rollout, verify the real user path and record what changed. That small habit turns a generic Kubernetes deployment guide into a dependable operating routine.