Kubernetes Liveness & Readiness Probe with Spring boot

SpringBoot Application Liveness&Readiness Probe 적용하기

Liveness&Readiness Probe 정의

The Liveness state of an application tells whether the internal state is valid. If Liveness is broken, this means that the application itself is in a failed state and cannot recover from it. In this case, the best course of action is to restart the application instance. For example, an application relying on a local cache should fail its Liveness state if the local cache is corrupted and cannot be repaired.

Liveness probe는 주기적으로 Pod를 감시하여 unhealthy상태의 Pod를 종료시킨 뒤 새로운 Pod를 재시작하고, Readiness probe 역시 주기적으로 Pod를 감시하여 unhealthy상태의 Pod를 routing 대상에서 제외시킨다.

Spring Boot Application의 Actuator를 통해 Liveness&Readiness 상태체크

상태를 체크하기위해 spring boot application의 actuator를 추가하고 k8s의 deployment에 probe를 추가하여 pod에 떠있는 application의 상태를 체크한다.

Probe Type에 따른 health check 방법

• exec
    Executes a specified command inside the container. The diagnostic is considered successful if the command exits with a status code of 0.
• grpc
    Performs a remote procedure call using gRPC. The target should implement gRPC health checks. The diagnostic is considered successful if the status of the response is SERVING.gRPC probes are an alpha feature and are only available if you enable the GRPCContainerProbe feature gate.
• httpGet
    Performs an HTTP GET request against the Pod's IP address on a specified port and path. The diagnostic is considered successful if the response has a status code greater than or equal to 200 and less than 400.
• tcpSocket
    Performs a TCP check against the Pod's IP address on a specified port. The diagnostic is considered successful if the port is open. If the remote system (the container) closes the connection immediately after it opens, this counts as healthy.

deployment.yaml

readinessProbe:
    failureThreshold: 3
    successThreshold: 1
    httpGet:
        path: /actuator/health/readiness
        port: 8080
        scheme: HTTP
    initialDelaySeconds: 200
    periodSeconds: 5
livenessProbe:
    failureThreshold: 3
    successThreshold: 1
    httpGet:
        path: /actuator/health/liveness
        port: 8080
        scheme: HTTP
    initialDelaySeconds: 600
    periodSeconds: 5

SpringBoot Actuator Dependency 추가 및 설정

build.gradle

implementation("org.springframework.boot:spring-boot-starter-actuator")

application.yaml

management:
  endpoints:
    web:
      base-path: /actuator
      exposure:
        include: health, info, metrics, prometheus
  endpoint:
    health:
      probes:
        enabled: true
      show-details: always
      status:
        http-mapping:
          DOWN: 503
          OUT_OF_SERVICE: 503
          UNKNOWN: 200
          UP: 200
        order: DOWN, OUT_OF_SERVICE, UNKNOWN, UP

Probe 조회 시 현재 구동중인 application과 외부시스템의 health check를 통해 status 주요도에 따른 상태전이를 ordering 할 수 있다. 예를들어 healthcheck 시 여러 서비스의 상태를 가지고있다면 order에 정의한 순서에 따라
최종상태가 결정된다 (DOWN, OUT_OF_SERVICE, UNKNOWN, UP의 상태를 가지고있다면 최종상태는 DOWN)
또한 각 상태의 http response code와 mapping하여 k8s의 probe의 status상태전이에 영향을 줄 수 있다.
(httpGet의 경우 200이상 400미만이 healthy, 외의 상태는 unhealthy)

외부 시스템 상태의 영향을 받지않고 순수히 application의 상태만 check하려면 liveness, readiness url을 각각 호출하여 검증하면된다.

References

• https://spring.io/blog/2020/03/25/liveness-and-readiness-probes-with-spring-boot
• https://kubernetes.io/docs/tasks/configure-pod-container/configure-liveness-readiness-startup-probes/
• https://kubernetes.io/docs/concepts/workloads/pods/pod-lifecycle/