After I’ve set up my DigitalOcean environment, I show how to launch a CoreOS cluster on it. To remain as relevant to productive experience as possible, I decided to run a Testing Cluster as described in the CoreOS documentation.

The first droplet is my etcd machine. Basically it runs a single etcd service, which is used as backend for fleet on all machines of my cluster. It was created with the following user-data:

user-data
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
#cloud-config

coreos:
etcd:
addr: $public_ipv4:4001
peer-addr: $public_ipv4:7001
fleet:
public-ip: $public_ipv4
metadata: "disk=ssd,platform=digitalocean,role=services"
units:
- name: etcd.service
command: start
- name: fleet.service
command: start
- name: docker-tcp.socket
command: start
enable: true
content: |
[Unit]
Description=Docker Socket for the API

[Socket]
ListenStream=$public_ipv4:2375
BindIPv6Only=both
Service=docker.service

[Install]
WantedBy=sockets.target
write_files:
- path: /etc/profile.d/aliases.sh
permissions: 0644
owner: root
content: |
alias ll='ls -l'

The exposed docker socket is not relevant for our cluster to function. I just used it to test docker images before deploying them to the whole cluster.

After the etcd droplet is running, fleetctl shows one machine with role=service as metadata. The metadata is used by fleet to decide, where clustered services should be started. I use role to distinguish between the one services machine and all other worker machines.

1
2
3
$ fleetctl list-machines
MACHINE IP METADATA
cfb065fa... 46.101.xxx.xxx disk=ssd,platform=digitalocean,role=services

To create the worker machines I need to know the IP of the etcd droplet just created and insert it in the user-data of the workers. With that, workers can be created as many as I like, using the following user-data:

user-data
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
#cloud-config

coreos:
fleet:
public-ip: $public_ipv4
metadata: "disk=ssd,platform=digitalocean,role=worker"
etcd_servers: "http://46.101.xxx.xxx:4001"
units:
- name: etcd.service
mask: true
- name: fleet.service
command: start
- name: setup-etcdctl-env.service
command: start
content: |
[Unit]
Description=Setup etcdctl env vars
Wants=coreos-setup-environment.service
After=coreos-setup-environment.service

[Service]
Type=oneshot
RemainAfterExit=true
ExecStart=/bin/bash -c "source /etc/profile.d/etcdctl.sh; env | grep -E 'ETCDCTL|FLEETCTL|INSTANCE_ID' >> /etc/environment"
[Install]
WantedBy=multi-user.target
write_files:
- path: /etc/profile.d/etcdctl.sh
permissions: 0644
owner: core
content: |
export ETCDCTL_PEERS="http://46.101.xxx.xxx:4001"
- path: /etc/profile.d/fleetctl.sh
permissions: 0644
owner: core
content: |
export FLEETCTL_ENDPOINT=unix:///var/run/fleet.sock
export FLEETCTL_EXPERIMENTAL_API=true
write_files:
- path: /etc/profile.d/aliases.sh
permissions: 0644
owner: root
content: |
alias ll='ls -l'

Instead of running etcd on every worker, I configured fleet to use the one from my etcd droplet. The setup-etcdctl-env.service makes it possible to use the etcdctl command from within fleet unit files, which I’ll describe in the next post.

To do failover testing with fleet I needed at least two workers.

1
2
3
4
5
$ fleetctl list-machines
MACHINE IP METADATA
9c4f0453... 46.101.xxx.xxx disk=ssd,platform=digitalocean,role=worker
c0a4dd67... 46.101.xxx.xxx disk=ssd,platform=digitalocean,role=worker
cfb065fa... 46.101.xxx.xxx disk=ssd,platform=digitalocean,role=services

I can access each of them by:

1
2
3
$ fleetctl ssh cf
$ fleetctl ssh c0
$ fleetctl ssh 9

Cheers,
visit