Liveness probes restart healthy pods. Readiness probes return 200 while the database is unreachable. Both failure modes are common, and both are avoidable.
Probes are three lines of YAML and they decide whether your service stays up. They are also, in our experience, the most consistently misconfigured thing in a Kubernetes manifest.
Mistake one: readiness that only proves the process started
A readiness endpoint returning a hardcoded 200 tells you the HTTP server is listening. It says nothing about whether the database is reachable. Traffic gets routed to a pod that cannot serve it, and the load balancer reports everything healthy while users see errors. Make readiness check the dependencies the request path actually needs.
Mistake two: liveness pointed at the same endpoint
If liveness also checks the database, a database blip restarts every pod simultaneously — turning a recoverable dependency failure into a full outage with a cold-start stampede on the other side. Liveness should answer one question: is this process wedged? Usually that is a trivial in-process check.
Mistake three: timings copied from a blog post
- initialDelaySeconds shorter than real startup causes restart loops under load
- failureThreshold of 1 makes a single slow response fatal
- periodSeconds too aggressive turns the probe itself into load
- No startupProbe on slow-booting applications, so liveness kills them before they are ready
Set these from measured startup behaviour, not defaults. Then test them by actually breaking the dependency in a non-production environment and watching what the cluster does.