Add Healthcheck for Containers

Step 1 - Creating Service

The new Healthcheck functionality is created as an extension to the Dockerfile and defined when a Docker image is built.

Create HTTP Service with a Healthcheck

The Dockerfile below extends an existing HTTP service and adds a healthcheck.

The healthcheck will curl the HTTP server running every second to ensure it's up. If the server responds with a non-200 request, curl will fail and an exit code 1 will be returned. After three failures, Docker will mark the container as unhealthy.

The format of the instruction is HEALTHCHECK [OPTIONS] CMD command.

FROM katacoda/docker-http-server:health
HEALTHCHECK --timeout=1s --interval=1s --retries=3 \
  CMD curl -s --fail http://localhost:80/ || exit 1

Currently, Healthcheck supports three different options:

interval=DURATION (default: 30s). This is the time interval between executing the healthcheck.

timeout=DURATION (default: 30s). If the check does not finish before the timeout, consider it failed.

retries=N (default: 3). How many times to recheck before marking a container as unhealthy.

The command executing must be installed as part of the container deployment. Under the covers, Docker will use docker exec to execute the command.

Build and Run

Before continuing, build and run the HTTP service.

docker build -t http .

By default it will start in a healthy state.

docker run -d -p 80:80 --name srv http

In the next steps we'll cause the HTTP Server to start throwing errors.

Step 2 - Crash Service

With the HTTP server running as a container, the Docker Daemon will automatically check the healthcheck based on the options. It will return the status when you list all the running containers, for example docker ps.

Set Unhealthy

The HTTP server has a special endpoint which will cause it to start reporting errors.

Make a http request to curl http://docker/unhealthy

The service will now go into error mode. In the next step, we'll look at how Docker handles this.

Step 3 - Verify Status

As the HTTP server is in an error state, the healthcheck should fail. Docker will report this as part of the metadata.

Detecting Errors

Docker will report the health status in various different places. To get the raw text stream, useful during automation, use Docker Inspect to pull out the Health Status field.

docker inspect --format "{{json .State.Health.Status }}" srv

The Health state stores a log of all the failures and any output from the command. This is useful for debugging why a container is considered unhealthy.

docker inspect --format "{{json .State.Health }}" srv

The status of all the containers can be viewed using docker ps

Step 4 - Fix Service

Use an extra HTTP endpoint to make the service healthy again. curl http://docker/healthy

View Healthy Status

Once the service is healthy again, Docker will update the status.

docker ps

docker inspect --format "{{json .State.Health.Status }}" srv

Step 5 - Healthchecks with Swarm

Docker Swarm can use these health checks to understand when services need to be restarted/recreated.

Initialise a Swarm cluster and deploy the newly created image as a service with two replicas.

docker rm -f $(docker ps -qa); 
docker swarm init
docker service create --name http --replicas 2 -p 80:80 http

You should see two containers responding curl docker

Randomly cause one of the nodes to be unhealthy with curl docker/unhealthy

You should only see one node processing requests as Swarm has automatically removed it from the load balancer: curl docker

Swarm will now restart the unhealthy service automatically. docker ps

After Swarm has restarted the service you should see two nodes again: curl docker

cd <directory>	Change directory
ls	List directory
echo 'contents' > <file>	Write contents to a file
cat <file>	Output contents of file

i	In Command Mode	Change into Insert Mode, you can now insert and edit text in the file
esc	In Insert Mode	Change into Command Mode, you can now execute commands
:wq	In Command Mode	Save and Exit

Welcome!

Docker Orchestration - Add Healthcheck for Containers

Congratulations!

You've completed the scenario!

Scenario Rating

Steps