Etcd Operator

Difficulty: beginner

Estimated Time: 30 minutes

The etcd operator manages etcd clusters deployed to Kubernetes and automates tasks related to operating an etcd cluster.

There are more spec examples on setting up clusters with different configurations

Read Best Practices for more information on how to better use etcd operator.

Read RBAC docs for how to setup RBAC rules for etcd operator if RBAC is in place.

Read Developer Guide for setting up a development environment if you want to contribute.

See the Resources and Labels doc for an overview of the resources created by the etcd-operator.

For more information, check out the links below:

Git Hub

Operator-Framework: https://github.com/operator-framework
Operator-SDK: https://github.com/operator-framework/operator-sdk/

Chat

Kubernetes Slack Chat (upstream): #kubernetes-operators at https://kubernetes.slack.com/
Operator-Framework on Google Groups: https://groups.google.com/forum/#!forum/operator-framework
OpenShift Operators Special Interest Group (SIG): https://commons.openshift.org/sig/OpenshiftOperators.html

Your environment is currently being packaged as a Docker container and the download will begin shortly. To run the image locally, once Docker has been installed, use the commands

cat scrapbook_openshift/operatorframework_etcd-operator-311_container.tar | docker load

docker run -it /openshift/operatorframework_etcd-operator-311:20210410

Restart Scenario

Next Scenario

Step 1 of 8

Creating the Custom Resource Definition (CRD)

Let's begin by creating a new project called myproject:

oc new-project myproject

Create the Custom Resource Definition (CRD) for the Etcd Operator:

cat > etcd-operator-crd.yaml<<EOF
apiVersion: apiextensions.k8s.io/v1beta1
kind: CustomResourceDefinition
metadata:
  name: etcdclusters.etcd.database.coreos.com
spec:
  group: etcd.database.coreos.com
  names:
    kind: EtcdCluster
    listKind: EtcdClusterList
    plural: etcdclusters
    shortNames:
    - etcdclus
    - etcd
    singular: etcdcluster
  scope: Namespaced
  version: v1beta2
  versions:
  - name: v1beta2
    served: true
    storage: true
EOF

oc create -f etcd-operator-crd.yaml

Verify the CRD was successfully created.

oc get crd

Creating the Service Account, Role, and RoleBinding

Create a dedicated Service Account responsible for running the Etcd Operator:

cat > etcd-operator-sa.yaml<<EOF
apiVersion: v1
kind: ServiceAccount
metadata:
  name: etcd-operator-sa
EOF

oc create -f etcd-operator-sa.yaml

Verify the Service Account was successfully created:

oc get sa

Create the Role that the etcd-operator-sa Service Account will need for authorization to perform actions against the Kubernetes API:

cat > etcd-operator-role.yaml<<EOF
apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
  name: etcd-operator-role
rules:
- apiGroups:
  - etcd.database.coreos.com
  resources:
  - etcdclusters
  - etcdbackups
  - etcdrestores
  verbs:
  - '*'
- apiGroups:
  - ""
  resources:
  - pods
  - services
  - endpoints
  - persistentvolumeclaims
  - events
  verbs:
  - '*'
- apiGroups:
  - apps
  resources:
  - deployments
  verbs:
  - '*'
- apiGroups:
  - ""
  resources:
  - secrets
  verbs:
  - get
EOF

oc create -f etcd-operator-role.yaml

Verify the Role was successfully created:

oc get roles

Create the RoleBinding to bind the etcd-operator-role Role to the etcd-operator-sa Service Account:

cat > etcd-operator-rolebinding.yaml<<EOF
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
  name: etcd-operator-rolebinding
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: Role
  name: etcd-operator-role
subjects:
- kind: ServiceAccount
  name: etcd-operator-sa
  namespace: myproject
EOF

oc create -f etcd-operator-rolebinding.yaml

Verify the RoleBinding was successfully created:

oc get rolebindings

Creating the Etcd Operator Deployment

Create the Deployment containing the Etcd Operator container image:

cat > etcd-operator-deployment.yaml<<EOF
apiVersion: extensions/v1beta1
kind: Deployment
metadata:
  labels:
    name: etcdoperator
  name: etcd-operator
spec:
  replicas: 1
  selector:
    matchLabels:
      name: etcd-operator
  template:
    metadata:
      labels:
        name: etcd-operator
    spec:
      containers:
      - command:
        - etcd-operator
        - --create-crd=false
        env:
        - name: MY_POD_NAMESPACE
          valueFrom:
            fieldRef:
              apiVersion: v1
              fieldPath: metadata.namespace
        - name: MY_POD_NAME
          valueFrom:
            fieldRef:
              apiVersion: v1
              fieldPath: metadata.name
        image: quay.io/coreos/[email protected]:c0301e4686c3ed4206e370b42de5a3bd2229b9fb4906cf85f3f30650424abec2
        imagePullPolicy: IfNotPresent
        name: etcd-operator
      serviceAccountName: etcd-operator-sa
EOF

oc create -f etcd-operator-deployment.yaml

Verify the Etcd Operator Deployment was successfully created:

oc get deploy

Verify the Etcd Operator Deployment pods are running:

oc get pods

Open a new terminal window to follow Etcd Operator logs in real-time:

export ETCD_OPERATOR_POD=$(oc get pods -l name=etcd-operator -o jsonpath='{.items[0].metadata.name}')
oc logs $ETCD_OPERATOR_POD -f

Observe the leader-election lease on the Etcd Operator Endpoint:

oc get endpoints etcd-operator -o yaml

Creating the EtcdCluster Custom Resource (CR)

Create an Etcd cluster by referring to the new Custom Resource, EtcdCluster, defined in the Custom Resource Definition on Step 1:

cat > etcd-operator-cr.yaml<<EOF
apiVersion: etcd.database.coreos.com/v1beta2
kind: EtcdCluster
metadata:
  name: example-etcd-cluster
spec:
  size: 3
  version: 3.1.10
EOF

oc create -f etcd-operator-cr.yaml

Verify the cluster object was created:

oc get etcdclusters

Watch the pods in the Etcd cluster get created:

oc get pods -l etcd_cluster=example-etcd-cluster -w

Verify the cluster has been exposed via a ClusterIP service:

oc get services -l etcd_cluster=example-etcd-cluster

Interacting with a Live Etcd Cluster

Let's now create another pod and attempt to connect to the etcd cluster via etcdctl:

oc run etcdclient --image=busybox busybox --restart=Never -- /usr/bin/tail -f /dev/null

Access the pod:

oc rsh etcdclient

Install the Etcd Client:

wget https://github.com/coreos/etcd/releases/download/v3.1.4/etcd-v3.1.4-linux-amd64.tar.gz
tar -xvf etcd-v3.1.4-linux-amd64.tar.gz
cp etcd-v3.1.4-linux-amd64/etcdctl .

Set the etcd version and endpoint variables:

export ETCDCTL_API=3
export ETCDCTL_ENDPOINTS=example-etcd-cluster-client:2379

Attempt to write a key/value into the Etcd cluster:

./etcdctl put operator sdk
./etcdctl get operator

Exit out of the client pod:

exit

Scaling and Modifying the Etcd Version

Let's change the size of the Etcd example-etcd-cluster CR. The Etcd Operator pod will detect the CR spec.size change and modify the number of pods in the cluster:

oc patch etcdcluster example-etcd-cluster --type='json' -p '[{"op": "replace", "path": "/spec/size", "value":5}]'

Let's change the version of our example-etcd-cluster CR. The etcd-operator pod will detect the CR spec.version change and create a new cluster with the newly specified image:

oc patch etcdcluster example-etcd-cluster --type='json' -p '[{"op": "replace", "path": "/spec/version", "value":3.2.13}]'

Disaster Recovery in Action

In another session delete a pod from the cluster and watch recovery in real-time.

export EXAMPLE_ETCD_CLUSTER_POD=$(oc get pods -l app=etcd -o jsonpath='{.items[0].metadata.name}')
oc delete pod $EXAMPLE_ETCD_CLUSTER_POD

Clean Up

Delete your Etcd cluster:

oc delete etcdcluster example-etcd-cluster

Delete the Etcd Operator:

oc delete deployment etcd-operator

Delete the Etcd CRD:

oc delete crd etcdclusters.etcd.database.coreos.com

Terminal

Dashboard

cd <directory>	Change directory
ls	List directory
echo 'contents' > <file>	Write contents to a file
cat <file>	Output contents of file

i	In Command Mode	Change into Insert Mode, you can now insert and edit text in the file
esc	In Insert Mode	Change into Command Mode, you can now execute commands
:wq	In Command Mode	Save and Exit

Welcome!

Etcd Operator

Congratulations!

You've completed the scenario!

Scenario Rating

Git Hub

Chat

Steps

Etcd Operator

Creating the Custom Resource Definition (CRD)

Creating the Service Account, Role, and RoleBinding

Creating the Etcd Operator Deployment

Creating the EtcdCluster Custom Resource (CR)

Interacting with a Live Etcd Cluster

Scaling and Modifying the Etcd Version

Disaster Recovery in Action

Clean Up

Welcome!

Etcd Operator

Congratulations!

You've completed the scenario!

Scenario Rating

Git Hub

Chat

Steps

Etcd Operator

Creating the Custom Resource Definition (CRD)

Creating the Service Account, Role, and RoleBinding

Creating the Etcd Operator Deployment

Creating the EtcdCluster Custom Resource (CR)

Interacting with a Live Etcd Cluster

Scaling and Modifying the Etcd Version

Disaster Recovery in Action

Clean Up

Help

Vim

Welcome! Connect to start

...or sign up using your email

Sign up using your email

You’ll love Katacoda

Guided Path

Learn By Doing

Stay up-to-date

Welcome back! Connect to start

...or log in using your email

Log in using your email

You’ll love Katacoda

Guided Path

Learn By Doing

Stay up-to-date

Welcome! Enter your email address to connect

😺

Have some feedback for us? Please tell us what's happening so we can improve Katacoda.

Creating Katacoda Scenarios

Debugging Scenarios

Tips

Debug Information