Skip to main content

Docker Swarm Configuration for InkBlink

Requirements

Tailscale Download (MAC/PC) Download | Tailscale
Docker Swarm (PC) Windows | Docker Docs

Initializing Docker Swarm for Inkblink

All machines that run Inkblink must be in the same Tailscale network.

WINDOWS : If docker swarm cannot be initialized using the Tailscale IP address, run a separate Tailscale image using docker. See Run Tailscale Using Docker

# Print Tailscale IP
tailscale ip -4
# if running Tailscale on a separate image
# When Tailscale is running:
docker exec -it <tailscale-container-name> tailscale ip -4

Once you have found out what the Tailscale IP address for your machine is, initialize a docker swarm using that address

# Initialize Docker Swarm
docker swarm init --advertise-addr <YOUR_TAILSCALE_IP> --listen-addr 0.0.0.0:2377

To verify the swarm has been initialized successfully:

$ netstat -an | grep 2377
  TCP    [::1]:2377             [::]:0                 LISTENING

A swarm has been initialized. For other nodes to join the swarm, each node must copy and paste the swarm token in their respective terminals. The token can be retrieved by the manager of the swarm using :

# The swarm manager can view nodes and assign nodes tasks
docker swarm join-token worker|manager

Expected Output:

$ docker swarm join-token worker
To add a worker to this swarm, run the following command:

    docker swarm join --token SWMTKN-1-0xuent3dqa4feid2p8tfmtbncsbr7idxq1g682jdiiq9bi71ex3-boafyko6qug1khac6mi38uc17 <YOUR_TAILSCALE_IP>

To

view

joined

 

 

Docker Swarm + Tailscale

Duenodes to conflictsyour betweenswarm, docker swarm initializing a custom network as the listening address, a tailscale image will run in a container to provide access to host as a proxy.

Steps to Initialize Tailscale Proxy (One machine in swarm cluster)
docker compose -f tailscale-compose.yaml up -d
#IPv4 Address of Container Running Tailscale (not local machine ip)
#Use this IP to initialize Docker Swarm
docker exec tailscale-proxy tailscale ip -4

To verify docker swarm has been initiated, check if port 2377 is listening

 netstat -an | grep 2377

Expected Terminal Output

$ netstat -an | grep 2377
  TCP    [::1]:2377             [::]:0                 LISTENING
Initialize Docker Swarm
#$DOCKER_IP is the result of docker exec tailscale-proxy tailscale ip -4
docker swarm init --advertise-addr "$DOCKER_IP" --listen-addr 0.0.0.0:2377


Join Docker Swarm as Manager
docker swarm join --token SWMTKN-1-59rogaer609u1iaqugvorkz12bunzew6tqanvg7ufspfrw0s85-6x9qkhzat0artan0olngl3o2c 100.79.134.37:2377
Join Docker Swarm as Worker**
docker swarm join --token SWMTKN-1-1hpmu6wmnmodlfgyz7c43x0803cyj2wj77vx49nipeo5machk1-6yqgneioaiku3wz4whelm91gj 100.79.134.37:2377

You can verify the nodes that have joined the initialized swarm using :input:

# Only Manager Nodes can run this commandRun
docker node ls

# Expected Terminaloutput Output

$

docker 
node ls
ID                            HOSTNAME         STATUS    AVAILABILITY   MANAGER STATUS   ENGINE VERSION
nr29nq3n9jee55zz0h43r6kxdy5eyvalhfj1umrewfk1eqiqni *   docker-desktop   Ready     Active         Leader           27.5.1
nz59idgmk5grpkk9eepg42glo     docker-desktop   Ready     Active                          28.0.1

TheUsing IDthe columnlisted willIDs, bethe importantdocker swarm manager can now assign roles to each node.

Assigning Roles to Nodes

To assign roles to a node, the service that you want to assign must have a label in designatingcompose.yaml. whichA nodetypical willcompose.yaml runfile whichmay service.

look

like

Deploying the Distributed System

this:

docker  stackprimary_server:
    deployimage: server
    build:
      context: ./backend
    hostname: primary_server
    ports:
      - "8887:8887"
      - "5001:5001"
      - "5002:5002"
      - "5003:5003"
    environment:
      TAILSCALE_IP: ${PRIMARY_SERVER_IP}
      KAFKA_BOOTSTRAP_SERVERS: ${KAFKA_IP}:9092
      BACKUP_SERVER_1_IP: ${BACKUP_SERVER_1_IP}
      BACKUP_SERVER_2_IP: ${BACKUP_SERVER_2_IP}
      BACKUP_SERVER_3_IP: ${BACKUP_SERVER_3_IP}
      PRIMARY_SERVER_IP: ${PRIMARY_SERVER_IP}
      COORDINATOR_IP: ${COORDINATOR_IP}
    command: >
      sh -c compose.yaml"/wait-for-backups.sh inkblink&& mvn exec:java -Dexec.mainClass='com.server.WebServer' -Dexec.args='8887 5001 ${BACKUP_SERVER_1_IP}:6001,${BACKUP_SERVER_2_IP}:7001,${BACKUP_SERVER_3_IP}:4001 true'"
    networks:
      - my_network
    deploy:
      placement:
        constraints:
          - node.labels.role == main_services
        # Under deploy/placement/constraints, you can add a label to the service to assign later to a node

6Once servicesyou inhave totaladded runlabels into thiseach containerservice, (foryou twocan nodes):now update each node's role:

Service Name

Node Label

Kafka

main_services
Coordinatormain_services
Primaryprimary
Backup 1primary
Backup 2primary
Backup 3primary
# DesignateAssigning Nodea Labelrole to Node_ID
dockera node
ls # <--Only Toa listswarm NODE_IDsmanager can assign roles to nodes
docker node update --label-add role=main_services<LABEL_NAME> <NODE_ID> # <-- Designate Node Label to Node

Preliminary steps to initialize the service is now complete.

 

Building Docker Image for Inkblink

All nodes that will run the service must each have the same source files as the manager. Ensure that the source files are up to date with the manager and build the images using:

docker compose build

The worker node does not run the images independently. Once the swarm initiates the services, the workers will run the respective images automatically.

Deploying Containers using Docker Swarm 

The compose.yaml utilizes several variables that are imported from .env located in the same path as the compose file. The .env file must be updated before building and deploying. The .env file may look something like this:

# Replace with corresponding tailscale IPs 
KAFKA_IP=100.83.215.115
PRIMARY_SERVER_IP=100.83.215.115
BACKUP_SERVER_1_IP=100.83.215.115
BACKUP_SERVER_2_IP=100.83.215.115
BACKUP_SERVER_3_IP=100.83.215.115
COORDINATOR_IP=100.83.215.115

Update the values of these variables manually to each node's respective Tailscale IP address.

The compose.yaml file does not automatically import these vars when building! Run the following to create a new compose.yaml file

# Export vars in .env to local terminal
export $(grep -v '^#' .env | xargs)

# Create another compose file that manager will use to deploy
envsubst < compose.yaml > rendered_compose.yaml

 

The created rendered_compose.yaml file will be used to deploy the containers to the swarm. 

The swarm manager can now build the containers using:

# Build using new compose file
docker compose -f rendered_compose.yaml build

To now deploy the service:

# Deploy containers to swarm
docker stack deploy -c rendered-compose.yaml inkblink

Confirm that services are running using either Docker Desktop or the terminal:

$ docker ps 
CONTAINER ID   IMAGE                           COMMAND                   CREATED             STATUS                      PORTS      NAMES
4a2130cb26fb   client:latest                   "docker-entrypoint.s…"   About an hour ago   Up About an hour                       inkblink_client_1.1.u9sic4vg0y0ecuynpykhvnvx9
4246003ed693   server:latest                   "mvn exec:java -Dexe…"   About an hour ago   Up About an hour            8887/tcp   inkblink_backup_server_3.1.j7orcp0g7bgnzowew2d2pudm6
5a37d51f4152   server:latest                   "mvn exec:java -Dexe…"   About an hour ago   Up About an hour            8887/tcp   inkblink_backup_server_2.1.v3oyyx2phblj1v1rnnkebvi3f
9de19e542fd2   server:latest                   "mvn exec:java -Dexe…"   About an hour ago   Up About an hour            8887/tcp   inkblink_backup_server_1.1.qg62zpihy890nqlnl0mkmh73o
28eadabc00e6   server:latest                   "sh -c '/wait-for-ba…"   About an hour ago   Up About an hour (Paused)   8887/tcp   inkblink_primary_server.1.wyla6fgxpjhovzbriygfefs0v
af844889dc73   connection_coordinator:latest   "mvn exec:java -Dexe…"   About an hour ago   Up About an hour            8887/tcp   inkblink_connection_coordinator.1.y8voelddpfqjgll2t7a0koyrt
97285295c564   apache/kafka:latest             "/__cacert_entrypoin…"   About an hour ago   Up About an hour            9092/tcp   inkblink_kafka.1.vv48duzjxu7cx6snpcxperut5
4029d93bd800   tailscale/tailscale             "tailscaled"              23 hours ago        Up 2 hours                             tailscale-proxy

 

WINDOWS : Running Tailscale as a Docker Instance

Create a new compose file, tailscale-compose.yaml with the following contents

services:
  tailscale-proxy:
    container_name: tailscale-proxy
    image: tailscale/tailscale
    network_mode: "host"
    cap_add:
      - NET_ADMIN
      - SYS_MODULE
    environment:
      - TS_AUTHKEY=;
      - TS_STATE_DIR=/var/lib/tailscale
      - TS_EXTRA_ARGS=--hostname=${HOSTNAME:-proxy-node}
    volumes:
      - tailscale-state:/var/lib/tailscale
      - /dev/net/tun:/dev/net/tun    
    restart: unless-stopped
    command: tailscaled

volumes:
  tailscale-state:

Run the following

exportdocker $(grepcompose -vf '^#' .env | xargs)
envsubst < tailscale-compose.yaml |build
# Run the tailscale image in the background
docker stackcompose deployup -c - inkblinkd

 


Back to top