Docker Swarm – Single Node

Docker Swarm Logo

In the previous tutorial, we created one small service, and let it run in an isolated Docker container. In reality, your application might consist of many of different services. An e-commerce application encompasses services to register new customers, search for products, list products, show recommendations and so on. These services might even exist more than one time when they are heavily requested. So an application can be seen as a composition of different services (that run in containers).

In this first part of the tutorial, we will work with the simple application of the Docker Basics Tutorial, that contains only one service. We will deploy this service more than one time and let run on only one machine. In part II we will scale this application over many machines.

Prerequisites

Before we start, you should have completed the first part of the tutorial series. As a result, you should an image uploaded to the DockerHub registry. In my case, the image name is vividbreeze/docker-tutorial:version1.

Docker Swarm

As mentioned above, a real-world application consists of many containers spread over different hosts. Many hosts can be grouped to a so-called swarm (mainly hosts that run Docker in swarm-mode). A swarm is managed by one or more swarm managers and consists of one or many workers. Before we continue, we have to initial a swarm on our machine.

docker swarm init

Swarm initialized: current node (pnb2698sy8gw3c82whvwcrd77) is now a manager.

To add a worker to this swarm, run the following command:

    docker swarm join --token SWMTKN-1-39y3w3x0iiqppn57pf2hnrtoj867m992xd9fqkd4c3p83xtej0-9mpv98zins5l0ts8j62ociz4w 192.168.65.3:2377

To add a manager to this swarm, run 'docker swarm join-token manager' and follow the instructions.

The swarm was initialised with one node (our machine) as a swarm manager.

Docker Stack

We now have to design our application. We do this in a file called docker-compose.yml.  So far, we have just developed one service, and that runs inside one Docker container. In this part of the tutorial, our application will only consist of one service. Now let us assume this service is heavily used and we want to scale it.

version: "3"
services:
  dataservice:
    image: vividbreeze/docker-tutorial:version1
    deploy:
      replicas: 3
    ports:
      - "4000:8080"

The file contains the name of our service and the number of instances (or replicas) that should be deployed. We now do the port mapping here. The port 8080 that is used by the service inside of our container will be mapped to the port 4000 on our host.

To create our application use (you have to invoke this command from the vm-manager node)

docker stack deploy -c docker-compose.yml dataapp

Creating network dataapp_default
Creating service dataapp_dataservice

Docker now has created a network dataservice_web and a network dataservice_webnet. We will come to networking in the last part of this tutorial. By “stack”, Docker means a stack of (scaled) services that together form an application. A stack can be deployed on one swarm. It has to be called from a Swarm manager.

Let us now have a look, of how many containers were created

docker container ls

ONTAINER ID        IMAGE                                  COMMAND              CREATED                  STATUS              PORTS                    NAMES
bb18e9d71530        vividbreeze/docker-tutorial:version1   "java DataService"   Less than a second ago   Up 8 seconds                                 dataapp_dataservice.3.whaxlg53wxugsrw292l19gm2b
441fb80b9476        vividbreeze/docker-tutorial:version1   "java DataService"   Less than a second ago   Up 7 seconds                                 dataapp_dataservice.4.93x7ma6xinyde9jhearn8hjav
512eedb2ac63        vividbreeze/docker-tutorial:version1   "java DataService"   Less than a second ago   Up 6 seconds                                 dataapp_dataservice.1.v1o32qvqu75ipm8sra76btfo6

In Docker terminology, each of these containers is called a task. Now each container cannot be accessed directly through the localhost and the port (they have no port), but through a manager, that listens to port 4000 on the localhost. These five containers, containing the same service, are bundled together and appear as one service. This service is listed by using

docker service ls
ID                  NAME                  MODE                REPLICAS            IMAGE                                  PORTS
zfbbxn0rgksx        dataapp_dataservice   replicated          5/5                 vividbreeze/docker-tutorial:version1   *:4000->8080/tcp

You can see the tasks (containers) that belong to this services with

docker service ps dataservice_web

ID                  NAME                    IMAGE                                  NODE                    DESIRED STATE       CURRENT STATE            ERROR                         PORTS
lmw0gnxcs57o        dataapp_dataservice.1       vividbreeze/docker-tutorial:version1   linuxkit-025000000001   Running             Running 13 minutes ago
fozpqkmrmsb3        dataapp_dataservice.2       vividbreeze/docker-tutorial:version1   linuxkit-025000000001   Running             Running 13 minutes ago
gc6dccwxw53f        dataapp_dataservice.3       vividbreeze/docker-tutorial:version1   linuxkit-025000000001   Running             Running 13 minutes ago

Now let us call the service 10 times

repeat 10 { curl localhost:4000; echo } (zsh)
for ((n=0;n<10;n++)); do curl localhost:4000; echo; done (bash)

<?xml version="1.0" ?><result><name>hello</name><id>2925</id></result>
<?xml version="1.0" ?><result><name>hello</name><id>1624</id></result>
<?xml version="1.0" ?><result><name>hello</name><id>2515</id></result>
<?xml version="1.0" ?><result><name>hello</name><id>2925</id></result>
<?xml version="1.0" ?><result><name>hello</name><id>1624</id></result>
<?xml version="1.0" ?><result><name>hello</name><id>2515</id></result>
<?xml version="1.0" ?><result><name>hello</name><id>2925</id></result>
<?xml version="1.0" ?><result><name>hello</name><id>1624</id></result>
<?xml version="1.0" ?><result><name>hello</name><id>2515</id></result>
<?xml version="1.0" ?><result><name>hello</name><id>2925</id></result>

Now you can see, that our service is called ten times, each time one of the services running inside of the containers were used to handle the request (you see three different ids). The service manager (dataservice-web) acts as a load-balancer. In this case, the load balancer uses a round-robin strategy.

To sum it up, in the docker-compose.yml, we defined our desired state (3 replicas of one service). Docker tries to maintain this desired state using the resources that are available. In our case, one host (one node). A swarm-manager manages the service, including the containers, we have just created. The service can be reached at port 4000 on the localhost.

Restart Policy

This can be useful for updating the number of replicas or changing other parameters. Let us play with some of the parameters. Let us add a restart policy to our docker-compose.yml

version: "3"
services:
  dataservice:
    image: vividbreeze/docker-tutorial:version1
    deploy:
      replicas: 3
      restart_policy:
        condition: on-failure
    ports:
      - "4000:8080"

and update our configuration

docker stack deploy -c docker-compose.yml dataapp

Let us now call our service again 3 times to memorise the ids

repeat 3 { curl localhost:4000; echo  }

<?xml version="1.0" ?><result><name>hello</name><id>713</id></result>
<?xml version="1.0" ?><result><name>hello</name><id>1157</id></result>
<?xml version="1.0" ?><result><name>hello</name><id>3494</id></result>

Now let us get the names of our containers

docker container ls -f "name=dataservice_web"

CONTAINER ID        IMAGE                                  COMMAND              CREATED             STATUS              PORTS               NAMES
953e010ab4e5        vividbreeze/docker-tutorial:version1   "java DataService"   15 minutes ago      Up 15 minutes                           dataapp_dataservice.1.pb0r4rkr8wzacitgzfwr5fcs7
f732ffccfdad        vividbreeze/docker-tutorial:version1   "java DataService"   15 minutes ago      Up 15 minutes                           dataapp_dataservice.3.rk7seglslg66cegt6nrehzhzi
8fb716ef0091        vividbreeze/docker-tutorial:version1   "java DataService"   15 minutes ago      Up 15 minutes                           datasapp_dataservice.2.0mdkfpjxpldnezcqvc7gcibs8

Now let us kill one of these containers, to see if our manager will start a new one again

docker container rm -f 953e010ab4e5

It may take a few seconds, but then you will see a newly created container created by the swarm manager (the container-id of the first container is now different).

docker container ls -f "name=dataservice_web"

CONTAINER ID        IMAGE                                  COMMAND              CREATED             STATUS              PORTS               NAMES
bc8b6fa861be        vividbreeze/docker-tutorial:version1   "java DataService"   53 seconds ago      Up 48 seconds                           dataapp_dataservice.1.5aminmnu9fx8qnbzoklfbzyj5
f732ffccfdad        vividbreeze/docker-tutorial:version1   "java DataService"   17 minutes ago      Up 17 minutes                           dataapp_dataservice.3.rk7seglslg66cegt6nrehzhzi
8fb716ef0091        vividbreeze/docker-tutorial:version1   "java DataService"   18 minutes ago      Up 17 minutes                           dataapp_datavervice.2.0mdkfpjxpldnezcqvc7gcibs8

The id in the response of one of the replicas of the service has changed

repeat 3 { curl localhost:4000; echo  }

<?xml version="1.0" ?><result><name>hello</name><id>2701</id></result>
<?xml version="1.0" ?><result><name>hello</name><id>1157</id></result>
<?xml version="1.0" ?><result><name>hello</name><id>3494</id></result>

Resources Allocation

You can also change the resources, such as CPU-time and memory that will be allocated to a service

...
image: vividbreeze/docker-tutorial:version1
deploy:
  replicas: 3
  restart_policy:
    condition: on-failure
  resources:
    limits:
      cpus: '0.50'
      memory: 10M
    reservations:
      cpus: '0.25'
      memory: 5M
...

The service will be allocated to at most 50% CPU-time and 50 MBytes of memory, and at least 25% CPU-time and 5 MBytes of memory.

Docker Compose

Instead of docker stack, you can also use docker-compose. docker-compose is a program, written in Python, that does the container orchestration for you on a local machine, e.g. it ignores the deploy-part in the docker-compose.yml.

However, docker-compose uses some nice debugging and clean-up functionality, e.g. if you start our application with

docker-compose -f docker-compose.yml up

you will see the logs of all services (we only have one at the moment) colour-coded in your terminal window.

WARNING: Some services (web) use the 'deploy' key, which will be ignored. Compose does not support 'deploy' configuration - use `docker stack deploy` to deploy to a swarm.
WARNING: The Docker Engine you're using is running in swarm mode.

Compose does not use swarm mode to deploy services to multiple nodes in a swarm. All containers will be scheduled on the current node.

To deploy your application across the swarm, use `docker stack deploy`.

Creating network "docker_default" with the default driver
Pulling web (vividbreeze/docker-tutorial:version1)...
version1: Pulling from vividbreeze/docker-tutorial
Digest: sha256:39e30edf7fa8fcb15112f861ff6908c17d514af3f9fcbccf78111bc811bc063d
Status: Downloaded newer image for vividbreeze/docker-tutorial:version1
Creating docker_web_1 ... done
Attaching to docker_web_1

You can see in the warning, that the deploy part of your docker-compose.yml is ignored, as docker-compose focusses on the composition of services on your local machine, and not across a swarm.

If you want to clean up (containers, volumes, networks and other) just use

docker-compose down

docker-compose also allows you to build your images (docker stack won’t) in case it hasn’t been built before, e.g.

build:
  context: .
  dockerfile: Dockerfile.NewDataservice
image: dataserviceNew

You might notice on the output of many commands, that docker-compose is different from the Docker commands. So again, use docker-compose only for Docker deployments on one host or to verify a docker-compose.yml on your local machine before using it in production.

Further Remarks

To summarise

  • Use docker swarm to define a cluster that runs our application. In our case the swarm consisted only of one machine (no real cluster). In the next part of the tutorial, we will see that a cluster can span various physical and virtual machines.
  • Define your application in a docker-compose.yml.
  • Use docker stack to deploy your application in the swarm in a production environment or docker-compose to test and deploy your application in a development environment.

Of course, there is more to Services, than I explained in this tutorial. However, I hope it helped as a starting point to go into more depth.

 

Chris

My background lies in software-development and software-architecture. I have been working for over 20 years for national and multi-national organisations in large and small teams. In 2006 I came in touch with agile values and principles. Since then I am applying these values on a daily basis as an Agile Coach and Scrum Master. However, I still can’t detach myself from software-development. Hence, I develop some minor projects or experiment with technology I hear from the teams I work with.