Docker container peer IP discovery with Swarm services

 

When running applications deployed as a docker “service” within Docker Swarm (post 1.12 swarm mode), often those application’s have the need to discover their peer containers for purposes such as forming a cluster. Similar to “service discovery”, I’ll just use the term “peer discovery” for this use-case.

The simple example is as follows: You create a swarm service bound to a docker network “N” and launch/scale X containers over it. How do those container instances find one another on the docker network?

As it stands today, simple dynamic discovery of peer container instances within a named swarm “service” residing on a particular docker network is not trivial. The Docker engine’s service API, can provide the information you need to figure this out, however this still puts a lot of work on you, the container developer who needs to collect this info and then act upon it in order to form a higher level cluster between your containers.

I had this specific challenge for a Java based service that needed to form a peer cluster dynamically. Out of that use case I came up with a generic library that you can drop into your Java container application called docker-discovery-swarm-service which is available at: https://github.com/bitsofinfo/docker-discovery-swarm-service

The purpose of this library is for “self-discovery” from within your JVM based Docker service application where you need to discover what your accessible docker service network bound IP is, as well as that of your peers within the same named docker service. As noted above this is critical if your container has to do further peer discovery for other services it provides or clustering groups it must form.

You can read all the details of how it works and how to use it here: https://github.com/bitsofinfo/docker-discovery-swarm-service

Hopefully it will be of use to you as well.

4 thoughts on “Docker container peer IP discovery with Swarm services

  1. Is their an example of how this can be configured from docker-compose file. Can you provide an example Java EE app that uses this on payara5 server.

      1. Does this work only on a core Java app run from the CLI or is a JavaEE app deployed on a docker swarm stack with docker stack deploy -c docker-compose.yml appname supported. I have added the maven dependency, establish listen on Unix socket added the docker run flags as labels within my compose file. The stack deploys the app with multiple instances but hazelcast is not discovering peers. Please can you point me in the right direction I may be missing a very important step.

        Must I include the hazelcast xml config file and some code in a my app onstart method to boostrap hazelcast? Any pointers would be highly appreciated

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s