Working around with Docker healthcheck logs

Docker health checks are a feature that allows you to monitor the status of a container and ensure that the services running inside it are operational. A health check defines a command that the Docker engine executes inside the container at regular intervals. The result of this command is used to determine whether the container is healthy or not.

Table of Contents

• Adding a Health Check to a Docker Container
  • Using Dockerfile
  • Using docker-compose.yml
• Checking Container Health Status
• Viewing Health Check Logs
• Best Practices
• Common Use Cases for Health Checks
  • Web Services
  • Database Services
  • API Endpoints
• Conclusion

Adding a Health Check to a Docker Container

Using Dockerfile

Here’s an example of how to add a health check in a Dockerfile:

FROM nginx:latest

# Add health check command
HEALTHCHECK --interval=30s --timeout=10s --retries=3 \
  CMD curl -f http://localhost/ || exit 1

In this example:

• --interval=30s: The time between running the check (30 seconds).
• --timeout=10s: The time after which the check is considered to have failed if no response is received (10 seconds).
• --retries=3: The number of consecutive failures needed before marking the container as unhealthy.

Using docker-compose.yml

In a docker-compose.yml file, you can add a health check like this:

services:
  web:
    image: nginx:latest
    healthcheck:
      test: curl -f http://localhost:8000/ || exit 1
      # or using wget
      # test: wget -nv -t1 --spider http://localhost || exit 1
      interval: 30s
      timeout: 10s
      retries: 3

Checking Container Health Status

Once you’ve set up the health check, Docker will automatically run it and update the health status of the container. You can check the status with the following command:

docker ps

This will show an additional column named STATUS with health status, e.g., healthy, unhealthy, or starting.

Viewing Health Check Logs

If you want to see the logs for a Docker container’s health check, you can inspect the container and look for health check details.

Here’s how you can do it:

Using docker inspect

docker inspect --format "{{json .State.Health }}" <container_name> | jq

This command will give you a JSON output containing detailed information about the health check, including the logs:

{
  "Status": "healthy",
  "FailingStreak": 0,
  "Log": [
    {
      "Start": "2024-08-08T10:00:00.000000000Z",
      "End": "2024-08-08T10:00:02.000000000Z",
      "ExitCode": 0,
      "Output": "Health check passed\n"
    },
    {
      "Start": "2024-08-08T09:55:00.000000000Z",
      "End": "2024-08-08T09:55:02.000000000Z",
      "ExitCode": 1,
      "Output": "curl: (7) Failed to connect to localhost port 80: Connection refused\n"
    }
  ]
}

• Status: Overall health status of the container (healthy, unhealthy, or starting).
• FailingStreak: Number of consecutive failed health checks.
• Log: List of recent health check logs, including timestamps, exit codes, and command output.

Example of Interpreting Logs

Suppose you have the following output:

{
  "Status": "unhealthy",
  "FailingStreak": 2,
  "Log": [
    {
      "Start": "2024-08-08T10:00:00.000000000Z",
      "End": "2024-08-08T10:00:02.000000000Z",
      "ExitCode": 1,
      "Output": "curl: (7) Failed to connect to localhost port 80: Connection refused\n"
    },
    {
      "Start": "2024-08-08T09:55:00.000000000Z",
      "End": "2024-08-08T09:55:02.000000000Z",
      "ExitCode": 1,
      "Output": "curl: (7) Failed to connect to localhost port 80: Connection refused\n"
    }
  ]
}

This tells us:

• The container is currently unhealthy.
• There have been 2 consecutive failures.
• The health check command failed to connect to localhost:80, indicating that the web server might not be running.

Best Practices

• Use health checks to monitor critical services within containers.
• Make health checks lightweight to avoid unnecessary overhead.
• Combine with orchestration tools like Docker Swarm or Kubernetes for automated recovery and scaling based on health status.

Common Use Cases for Health Checks

• Web Services: Ensure that a web server is responding to HTTP requests.
• Database Services: Verify database connectivity and responsiveness.
• API Endpoints: Check if an API is returning expected responses.

Web Services

FROM nginx:latest

# Add health check command
HEALTHCHECK --interval=30s --timeout=10s --retries=3 \
  CMD curl -f http://localhost/ || exit 1

services:
  web:
    image: nginx:latest
    healthcheck:
      test: curl -f http://localhost:8000/ || exit 1
      # or using wget
      # test: wget -nv -t1 --spider http://localhost || exit 1
      interval: 30s
      timeout: 10s
      retries: 3

Database Services

FROM mysql:latest

# Add health check command
HEALTHCHECK --interval=30s --timeout=10s --retries=3 \
  CMD mysql --user=user --password=password --execute='SELECT 1' --host=127.0.0.1 --port=3306

services:
  db:
    image: mysql:latest
    healthcheck:
      test: mysql --user=user --password=password --execute='SELECT 1' --host=127.0.0.1 --port=3306
      interval: 30s
      timeout: 10s
      retries: 3

API Endpoints

FROM app:latest

# Add health check command
HEALTHCHECK --interval=30s --timeout=10s --retries=3 \
  CMD curl -f http://localhost/api/v1/healthcheck || exit 1

services:
  app:
    image: app:latest
    healthcheck:
      test: curl -f http://localhost/api/v1/healthcheck || exit 1
      interval: 30s
      timeout: 10s
      retries: 3

Conclusion

Docker health checks are a valuable tool for ensuring that your containers and services are running smoothly. By understanding and utilizing these checks, you can increase the reliability and maintainability of your containerized applications.