After I’ve set up my DigitalOcean environment, I show how to launch a CoreOS cluster on it. To remain as relevant to productive experience as possible, I decided to run a Testing Cluster as described in the CoreOS documentation.

The first droplet is my etcd machine. Basically it runs a single etcd service, which is used as backend for fleet on all machines of my cluster. It was created with the following user-data:

user-data
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
#cloud-config

coreos:
  etcd:
    addr: $public_ipv4:4001
    peer-addr: $public_ipv4:7001
  fleet:
    public-ip: $public_ipv4
    metadata: "disk=ssd,platform=digitalocean,role=services"
  units:
    - name: etcd.service
      command: start
    - name: fleet.service
      command: start
    - name: docker-tcp.socket
      command: start
      enable: true
      content: |
        [Unit]
        Description=Docker Socket for the API

        [Socket]
        ListenStream=$public_ipv4:2375
        BindIPv6Only=both
        Service=docker.service

        [Install]
        WantedBy=sockets.target
write_files:
  - path: /etc/profile.d/aliases.sh
    permissions: 0644
    owner: root
    content: |
      alias ll='ls -l'

The exposed docker socket is not relevant for our cluster to function. I just used it to test docker images before deploying them to the whole cluster.

After the etcd droplet is running, fleetctl shows one machine with role=service as metadata. The metadata is used by fleet to decide, where clustered services should be started. I use role to distinguish between the one services machine and all other worker machines.

1
2
3
$ fleetctl list-machines
MACHINE		IP		METADATA
cfb065fa...	46.101.xxx.xxx	disk=ssd,platform=digitalocean,role=services

To create the worker machines I need to know the IP of the etcd droplet just created and insert it in the user-data of the workers. With that, workers can be created as many as I like, using the following user-data:

user-data
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
#cloud-config

coreos:
  fleet:
    public-ip: $public_ipv4
    metadata: "disk=ssd,platform=digitalocean,role=worker"
    etcd_servers: "http://46.101.xxx.xxx:4001"
  units:
    - name: etcd.service
      mask: true
    - name: fleet.service
      command: start
    - name: setup-etcdctl-env.service
      command: start
      content: |
          [Unit]
          Description=Setup etcdctl env vars
          Wants=coreos-setup-environment.service
          After=coreos-setup-environment.service

          [Service]
          Type=oneshot
          RemainAfterExit=true
          ExecStart=/bin/bash -c "source /etc/profile.d/etcdctl.sh; env | grep -E 'ETCDCTL|FLEETCTL|INSTANCE_ID' >>  /etc/environment"
          [Install]
          WantedBy=multi-user.target
write_files:
  - path: /etc/profile.d/etcdctl.sh
    permissions: 0644
    owner: core
    content: |
      export ETCDCTL_PEERS="http://46.101.xxx.xxx:4001"
  - path: /etc/profile.d/fleetctl.sh
    permissions: 0644
    owner: core
    content: |
      export FLEETCTL_ENDPOINT=unix:///var/run/fleet.sock
      export FLEETCTL_EXPERIMENTAL_API=true
write_files:
  - path: /etc/profile.d/aliases.sh
    permissions: 0644
    owner: root
    content: |
      alias ll='ls -l'

Instead of running etcd on every worker, I configured fleet to use the one from my etcd droplet. The setup-etcdctl-env.service makes it possible to use the etcdctl command from within fleet unit files, which I’ll describe in the next post.

To do failover testing with fleet I needed at least two workers.

1
2
3
4
5
$ fleetctl list-machines
MACHINE		IP		METADATA
9c4f0453...	46.101.xxx.xxx	disk=ssd,platform=digitalocean,role=worker
c0a4dd67...	46.101.xxx.xxx	disk=ssd,platform=digitalocean,role=worker
cfb065fa...	46.101.xxx.xxx	disk=ssd,platform=digitalocean,role=services

I can access each of them by:

1
2
3
$ fleetctl ssh cf
$ fleetctl ssh c0
$ fleetctl ssh 9

Cheers,
visit