Understanding Kubernetes Liveness Probes.

In a Kubernetes cluster, applications run in containers within Pods spread across various nodes. However, there are times when these applications may become unresponsive or experience startup delays, which can disrupt service availability. To address these issues, Kubernetes provides Liveness and Readiness Probes, enabling the kubelet to monitor the health of applications within containers and trigger restarts for any unresponsive ones. By using these probes, Kubernetes ensures applications remain reliable, responsive, and able to handle traffic, even if issues arise.

Imagine a scenario where a container has been running smoothly for some time, but suddenly, the application within becomes unresponsive, perhaps due to resource constraints. At this point, the container is no longer serving its purpose and needs a restart to restore availability. Rather than handling these restarts manually, Kubernetes offers a solution through Liveness Probes.

A Liveness Probe continuously monitors the health of an application; if it detects a failure, the kubelet steps in to automatically restart the affected container, ensuring minimal disruption. Kubernetes supports different types of Liveness Probes to support various application needs, which includes:

• Command-based Liveness Probes
• HTTP-based Liveness Checks
• TCP Socket Liveness Checks
• gRPC Liveness Probes

Each probe type provides a flexible way to keep applications responsive and reliable, even in the face of unexpected issues. In this article we will be discussing more on the different types for liveness probe.

Command-based liveness Probe

In Kubernetes this is a powerful mechanism used to monitor the health of containers by executing a command inside the container at specified intervals. This probe helps determine if the container is running properly and can automatically restart the container if it becomes unhealthy. This is particularly useful for applications that may not respond correctly but are still running.

How it works - The command specified in the liveness probe is executed inside the container. If the command exits with a status code of 0, the container is considered healthy. Conversely, if the command exits with a non-zero status, Kubernetes determines that the container is unhealthy and triggers a restart. This automatic restart can help maintain the reliability and availability of applications running in Kubernetes.

apiVersion: v1
kind: Pod
metadata:
  name: command-liveness-probe
spec:
  containers:
  - name: liveness-container
    image: k8s.gcr.io/busybox
    args:
    - /bin/sh
    - -c
    - "while true; do echo 'running'; sleep 5; done"
    livenessProbe:
      exec:
        command:
        - cat
        - /tmp/healthy
      initialDelaySeconds: 15
      periodSeconds: 5

Here, the probe checks for the existence of /tmp/healthy every 5 seconds, as specified by the periodSeconds parameter. The initialDelaySeconds parameter directs the kubelet to wait 15 seconds after the container starts before initiating the first probe.

The container’s startup commands first create /tmp/healthy and then delete it after 30 seconds. Once this file is deleted, the next probe will fail, triggering a restart of the container. The failureThreshold is set to 1, meaning the kubelet will consider the container unhealthy after just one failed probe and will restart it immediately.

Liveness HTTP Request

A Liveness HTTP Request Probe is a mechanism for monitoring the health of a containerized application by making HTTP requests to a specified endpoint within the container. This probe helps Kubernetes determine if the application is functioning correctly. If the application becomes unresponsive or fails to return a healthy status, Kubernetes automatically restarts the container, helping maintain application availability.

How it works — The liveness probe sends an HTTP GET request to a specified endpoint (e.g., /health) within the container. If the endpoint returns a success response (HTTP status 200-399), Kubernetes considers the application healthy. If the response indicates a failure (HTTP status 400 or higher, or no response), Kubernetes restarts the container, assuming it to be unhealthy. This enables automated recovery and resilience.

apiVersion: v1
kind: Pod
metadata:
  labels:
    test: liveness
  name: liveness-http
spec:
  containers:
  - name: liveness
    image: registry.k8s.io/e2e-test-images/agnhost:2.40
    args:
    - liveness
    livenessProbe:
      httpGet:
        path: /health
        port: 8080
        httpHeaders:
        - name: Custom-Header
          value: Awesome
      initialDelaySeconds: 3
      periodSeconds: 3

In the above example, the kubelet performs an HTTP GET request to the application’s /health endpoint on port 8080. If the response indicates a failure, the kubelet will restart the container; otherwise, the application is considered healthy.

TCP Liveness Probe

A TCP Liveness Probe in Kubernetes is used to monitor the health of an application by attempting to open a TCP connection to a specific port within the container. This probe is particularly useful for applications that may not expose HTTP endpoints but still need to be monitored for connectivity, such as databases, message brokers, or other services that communicate over TCP.

How it works — The kubelet sends a TCP request to the defined port in the container at regular intervals. If it can establish a connection, Kubernetes considers the application healthy. If the connection attempt fails, the kubelet marks the container as unhealthy and initiates a restart. This automatic restart process ensures that Kubernetes can maintain application availability even if the application stops accepting connections due to an error or a crash.

...
livenessProbe:
  tcpSocket:
    port: 8080
  initialDelaySeconds: 15
  periodSeconds: 5
...

gRPC Readiness Probe

gRPC Liveness Probe is designed to monitor the health of applications using the gRPC protocol, specifically those implementing the gRPC health checking protocol. It enables Kubernetes to check the health status of a gRPC service and restart the container if the application fails to respond correctly, helping maintain service availability.

How it works — The gRPC liveness probe sends a health-check request to a specified port in the container where the gRPC service is running. If the service responds as expected, Kubernetes considers the application healthy. If it fails to respond or indicates an unhealthy status, Kubernetes will mark the container as unhealthy and restart it. This probe type supports specifying a service name, which allows the use of the same port for both liveness and readiness probes.

...
    livenessProbe:
      grpc:
        port: 2379
      initialDelaySeconds: 10
...

Conclusion

Kubernetes liveness probes are essential for ensuring application health within a cluster. By using command, HTTP, TCP, and gRPC probes, you can monitor your applications effectively and automate restarts in case of failures. Understanding how to configure these probes allows for tailored health checks, improving application reliability and minimizing downtime. Embrace liveness probes to enhance your deployment strategies and foster a culture of resilience in your development practices.