Containers and Kubernetes: What They Actually Are and When You Actually Need Them
Containers and Kubernetes: What They Actually Are and When You Actually Need Them
Series: The Modern SDLC · Post 10 of 17 ← Post 9: Infrastructure as Code · Post 11: CD and GitOps →
Kubernetes has a reputation problem. Among engineers who've adopted it well, it's the platform that made their delivery process reliable and their infrastructure predictable. Among engineers who adopted it too early or without enough context, it's the thing that turned a straightforward deployment into a full-time operational job.
Both groups are describing the same technology. The difference is context: team size, workload characteristics, operational maturity, and — most importantly — whether the complexity Kubernetes introduces is justified by the problems it solves.
This post covers what containers and Kubernetes actually do, why they matter, and — as importantly — when they're the wrong tool for the job. Understanding the second part is what separates teams that get value from Kubernetes from teams that spend six months learning it before concluding they should have used something simpler.
The one thing to remember
A container is an immutable artefact. The exact same image that passes CI runs in staging and production. There is no deployment step that could introduce differences — you deploy the artefact, not instructions for building it.
This immutability is the foundational value of containerisation. Everything else — Kubernetes, orchestration, service meshes — builds on top of it.
What containers actually are
A container is a lightweight, isolated process that packages an application with everything it needs to run: runtime, libraries, configuration, and code. Unlike virtual machines, containers share the host operating system's kernel — which is why they start in milliseconds rather than minutes and use a fraction of the memory.
The practical consequence is environment parity. An application that runs in a container on a developer's laptop runs the same way in CI, in staging, and in production. The runtime version, the system libraries, the configuration — all identical, all specified in the Dockerfile. "Works on my machine" stops being possible because "my machine" and "production" run the same container image.
The image is the key concept. An image is a snapshot — a read-only, layered filesystem that contains everything the container needs. It's built once, pushed to a registry, and pulled by any environment that runs it. The image that passes your CI pipeline is the image that deploys to production. There's no build step on the production server, no dependency installation, no environment-specific setup. The artefact is the deployment.
Writing good Dockerfiles
A poorly written Dockerfile produces an image that's gigabytes in size, takes minutes to build, contains unnecessary build tools and source code, and has hundreds of CVEs from outdated base image packages. A well-written Dockerfile produces an image that's small, fast to build, and has minimal attack surface.
Multi-stage builds are the most impactful practice. Use one stage to compile or bundle — installing build tools, running the build, generating artefacts. Use a separate, minimal stage for the final image that contains only the runtime artefact. Build tools, source code, test files, and intermediate artefacts never appear in the image you ship.
A Node.js application that starts as 1.2GB using node:20 becomes 80MB using a multi-stage build with a distroless final image. The smaller image pulls faster, scans faster, starts faster, and has a smaller attack surface.
Choose the smallest base image that works. Three tiers:
Distroless (Google's approach) — no shell, no package manager, no operating system utilities. Just the runtime. Near-zero attack surface. The right choice when you don't need to shell into a container for debugging.
Alpine — 5MB, musl libc, has a shell. Most common choice for small images that still need basic tooling.
Debian slim — better compatibility for applications with complex native dependencies, slightly larger than Alpine.
Pin base images by digest, not by tag. FROM node:20-alpine is mutable — the image that tag points to can change without warning when the maintainer pushes an update. FROM node:20-alpine@sha256:abc123... is immutable — it always resolves to the same image bytes. Pinning by digest makes your builds reproducible and protects against upstream changes introducing vulnerabilities or breaking changes.
Layer order for cache efficiency. Docker caches layers and reuses them on subsequent builds if nothing above them has changed. Copy dependency files (package.json, requirements.txt, go.mod) and install dependencies before copying source code. Dependencies change rarely; source code changes constantly. A well-ordered Dockerfile turns an eight-minute build into a thirty-second one for typical source-only changes.
Run as a non-root user. The default in most base images is to run as root, which means a container escape gives an attacker root access to the host kernel surface. Add a non-root user and switch to it before the final CMD. One line that meaningfully reduces blast radius if the container is compromised.
.dockerignore prevents unnecessary files from being sent to the Docker build daemon: node_modules, .git, .env, test files, documentation. Smaller build context means faster builds and eliminates the risk of accidentally including secrets.
Kubernetes: what it does and what it costs
Kubernetes is a container orchestration platform. It solves the problem of running containers reliably at scale: scheduling them onto nodes, restarting failed ones, scaling them up and down, routing traffic, managing rolling deployments, and handling node failures transparently.
The value is real. Self-healing — containers that crash are restarted automatically. Declarative deployments — you declare "I want three replicas of this image" and Kubernetes continuously reconciles reality to match. Horizontal scaling — add CPU under load, remove it when traffic drops. Zero-downtime rolling updates — new pods pass health checks before old ones are terminated.
The cost is also real. Kubernetes has genuine operational overhead. Learning curve: a meaningful amount of time before a team is effective. Complexity: YAML manifests, networking abstractions, RBAC, storage classes, ingress controllers — each one a surface for misconfiguration. Debugging: a failed pod in Kubernetes has more places to look than a failed process on a server. Running it: even managed Kubernetes (EKS, GKE, AKS) requires understanding node management, cluster upgrades, and resource allocation.
When Kubernetes earns its complexity:
You have multiple services that need independent deployment cadences. You need reliable horizontal scaling based on load. You need to pack multiple workloads efficiently onto shared infrastructure. You have a team with the capacity to operate it well. You're running at a scale where the operational investment pays back in reliability and efficiency.
When Kubernetes is probably the wrong choice:
A single service with predictable load. A team of two or three who will spend more time learning Kubernetes than building the product. An early-stage product still finding product-market fit where infrastructure simplicity matters more than infrastructure power. An organisation without the platform engineering capacity to operate a cluster well.
For these situations, the honest alternatives are better: AWS App Runner, Railway, Render, or Fly.io for container deployment with dramatically less operational overhead. Earn Kubernetes by outgrowing simpler options — don't adopt it on principle.
The Kubernetes objects you'll actually use
Kubernetes has a large API surface, but most day-to-day work involves a small subset of it. These are the objects worth understanding thoroughly before worrying about the rest.
Pod — the smallest deployable unit. One or more containers sharing a network namespace. You rarely create Pods directly; higher-level controllers manage them.
Deployment — manages a set of identical Pods. Handles rolling updates, rollbacks, and replica counts. The standard way to run stateless services. Most of what you deploy will be a Deployment.
Service — a stable DNS name and load balancer for a set of Pods. Pods come and go; a Service provides a consistent address. ClusterIP for internal traffic, LoadBalancer for external traffic. Services are how services find each other — by name, not by IP.
Ingress — HTTP/HTTPS routing rules. Maps domain names and URL paths to Services. Requires an Ingress controller in the cluster (nginx-ingress, AWS ALB Ingress Controller, Traefik). The front door of your cluster for web traffic.
ConfigMap — non-secret configuration data injected as environment variables or mounted as files. Change a ConfigMap without rebuilding the image.
Secret — base64-encoded (not encrypted by default) sensitive data. Important caveat: Kubernetes Secrets are only as secure as your cluster's etcd encryption at rest and RBAC configuration. For production, use External Secrets Operator to sync from Vault or AWS Secrets Manager rather than managing secrets directly in Kubernetes.
HorizontalPodAutoscaler — scales Deployment replica count based on CPU, memory, or custom metrics. Scale out under load, scale in when traffic drops. KEDA extends this to event-driven scaling — scale to zero when a queue is empty.
NetworkPolicy — firewall rules for pod-to-pod traffic. Without NetworkPolicy, every pod in a cluster can reach every other pod — a compromised frontend pod can directly query your database. Default deny all, explicitly allow what's needed. One of the most impactful security improvements you can make to a cluster and one of the least commonly applied.
PodDisruptionBudget — guarantees minimum available replicas during node drains and cluster upgrades. Without it, Kubernetes can take down all pods of a service simultaneously during a node drain. With it, it's constrained to drain within the budget.
CronJob — run a container on a schedule. Database migrations before deployment, nightly report generation, data pipeline runs. The Kubernetes-native replacement for cron on a server.
Health checks: the detail that makes rolling deployments work
Kubernetes has two health check mechanisms. Understanding the difference between them is the detail that separates reliable rolling deployments from ones that route traffic to broken pods.
Readiness probes answer "is this pod ready to receive traffic?" A pod that fails its readiness probe is removed from the Service's load balancer — it stops receiving new requests — but it isn't restarted. This is what makes rolling deployments safe: new pods must pass readiness before old pods are terminated. During startup, during database connection establishment, during cache warming — the pod is alive but not ready.
Liveness probes answer "is this pod still functioning?" A pod that fails its liveness probe is restarted. This handles the crashed-but-still-running scenario — a deadlocked application that's technically alive but not processing requests.
The common mistake is using the same endpoint for both probes, or — worse — making the liveness probe check external dependencies. If your liveness probe calls the database and the database goes down, every pod in your deployment restarts simultaneously. The correct approach: liveness checks only internal application state, readiness checks everything the application needs to serve traffic.
Resource requests and limits: get these wrong and you'll know it
Every container in Kubernetes should have CPU and memory requests and limits set. Getting them wrong manifests in two common ways: pods evicted because the node runs out of resources (requests too low), or pods OOMKilled or CPU-throttled under normal load (limits too close to actual usage).
Requests are what the scheduler uses to place pods on nodes. Set them at the p50 of actual usage under normal load — what the container typically needs.
Limits are the maximum a container can consume. A container that exceeds its memory limit is killed. A container that exceeds its CPU limit is throttled (not killed). Set memory limits at the p99 with headroom. Set CPU limits higher than you think you need — CPU throttling is invisible but causes latency spikes that are hard to diagnose.
The starting point: deploy without limits, instrument with metrics, observe actual usage for a week, set requests at p50 and limits at p99 with 20% headroom. Revisit quarterly.
Helm: packaging Kubernetes manifests
Raw Kubernetes YAML is verbose, repetitive, and doesn't handle the difference between environments well. Helm is the package manager for Kubernetes — it wraps manifests in templates, provides a values system for environment-specific configuration, and manages versioned releases.
A Helm Chart packages all the Kubernetes manifests for an application with templating. The same chart deploys to dev (small instances, single replica) and production (larger instances, multiple replicas) by providing different values files. Third-party charts for common infrastructure — nginx-ingress, cert-manager, Prometheus — are available from public chart repositories and save significant work.
Helm vs Kustomize is a common decision point. Helm uses templating — Go templates with values injection. Kustomize uses overlays — a base configuration with environment-specific patches applied on top. They solve the same problem differently. Many teams use both: Helm for third-party charts (where the Chart is already written), Kustomize for internal service configuration (where you own the base manifests).
Managed Kubernetes: don't run your own control plane
Running your own Kubernetes control plane — managing etcd, the API server, the scheduler, and the controller manager — is a significant operational burden that the vast majority of teams shouldn't take on. Managed services handle control plane availability, upgrades, and security patching.
EKS (AWS) is the right choice for AWS-native organisations. IRSA (IAM Roles for Service Accounts) provides fine-grained IAM permissions per pod. AWS ALB Ingress Controller integrates naturally with load balancers. Fargate eliminates node management entirely for teams that prefer to pay per pod.
GKE (Google Cloud) is generally regarded as the most polished Kubernetes experience — Google invented Kubernetes and GKE reflects that depth. Autopilot mode removes node management entirely and charges per pod resource request. Strong multi-cluster tooling.
AKS (Azure) is the natural choice for Azure-native organisations. Strong Azure AD integration for authentication and authorisation.
For most teams: pick the managed service from your cloud provider and use managed node groups or Fargate/Autopilot. The operational investment in node management rarely produces returns that justify the overhead.
The production readiness checklist
The gap between "it runs in Kubernetes" and "it runs reliably in Kubernetes" is closed by working through this list before a service goes to production:
[ ] Resource requests AND limits set on every container
[ ] Liveness probe — restarts crashed or deadlocked containers
[ ] Readiness probe — removes unhealthy pods from load balancer
[ ] PodDisruptionBudget — survives node drains and upgrades
[ ] HorizontalPodAutoscaler — scales with load
[ ] Non-root user in Dockerfile
[ ] Read-only root filesystem where possible
[ ] NetworkPolicy — default deny, explicit allow
[ ] Image scanned for CVEs before push
[ ] Image signed and signature verified at admission
[ ] Secrets from vault, not baked into image or ConfigMap
[ ] Graceful shutdown handler (SIGTERM → drain → exit)
[ ] Structured logs to stdout, not to files
[ ] OpenTelemetry instrumented for traces and metrics
[ ] Replicas ≥ 2 for any service requiring high availability
What goes wrong when you get this wrong
Adopting Kubernetes before earning it. A two-engineer startup running twelve microservices on Kubernetes spends more time managing infrastructure than building the product. The operational overhead consumes capacity that should go to users.
Missing health checks. Rolling deployments without readiness probes route traffic to pods that aren't ready. Missing liveness probes leave deadlocked pods serving errors indefinitely. Both failures are silent until users notice.
No resource limits. A single poorly-configured pod consumes all memory on a node. Other pods are evicted. Cascading failures across services that happen to share the node.
No NetworkPolicy. A compromised frontend container has direct network access to the database, the secrets service, and every other pod in the cluster. One container compromise becomes a cluster-wide breach.
Kubernetes Secrets without encryption. Base64 is not encryption. Anyone with etcd access or appropriate RBAC permissions can read every Secret in the cluster. Encrypt etcd at rest. Better: use External Secrets Operator and never store sensitive values in Kubernetes Secrets at all.
If you do one thing from this post
If you're not yet containerised: write a Dockerfile for your application, build the image, and run it locally. Verify the application works identically inside the container as it does outside. That's step one — everything else follows from having an immutable, reproducible artefact.
If you're already containerised but considering Kubernetes: answer this question honestly first. What specific problem do you have today that Kubernetes solves and that a simpler platform doesn't? If the answer is specific and operational — "we need to pack multiple services efficiently onto shared infrastructure and autoscale based on queue depth" — proceed. If the answer is "we want to be cloud-native" or "everyone else is using it," use something simpler until you have the specific problem.
Next up: Post 11 — GitOps: Making Deployment So Boring It Never Wakes You Up at 3am
← Post 9: Infrastructure as Code: Treat Your Cloud Like a Codebase
