In this step, we will create a Portworx volume (PVC) for MySQL.

Step: Create StorageClass and PersistentVolumeClaim

Take a look at the StorageClass definition for Portworx

cat px-repl3-sc.yaml

The parameters are declarative policies for your storage volume. See here for a full list of supported parameters.

Create the storage class using:

kubectl create -f px-repl3-sc.yaml

Take a look at the Persistent Volume Claim

cat px-mysql-pvc.yaml

This defines the maximum volume size. Portworx will thin provision the volume and let it grow to 2GB size.

Create the PersistentVolumeClaim using:

kubectl create -f px-mysql-pvc.yaml

Now that we have the volumes created, let's deploy MySQL !

In this step, we will deploy the MySQL application using the PersistentVolumeClaim created before.

Step: Create secret for MySQL

Below we are creating a Secret to store the mysql password.

echo -n mysql123 > password.txt
kubectl create secret generic mysql-pass --from-file=password.txt

Below we will create a mysql Deployment that uses a Portworx PVC.

Take a look at the yaml: cat mysql-app.yaml

Observe the volumeMounts and volumes sections where we mount the PVC.

Now use kubectl to deploy MySQL.

kubectl create -f mysql-app.yaml

Step: Verify MySQL pod is ready

Below commands wait till the mysql pod is in ready state.

watch kubectl get pods -l app=mysql -o wide

When the pod is in Running state then then hit clear to ctrl-c and clear the screen.

In this step, we will use pxctl to inspect the volume

Step: Inspect the Portworx volume

Portworx ships with a pxctl command line that can be used to manage Portworx.

Below we will use pxctl to inspect the underlying volume for our PVC.

VOL=`kubectl get pvc | awk '/^px-mysql-pvc/{print $3}'`
PX_POD=$(kubectl get pods -l name=portworx -n kube-system -o jsonpath='{.items[0].metadata.name}')
kubectl exec -it $PX_POD -n kube-system -- /opt/pwx/bin/pxctl volume inspect ${VOL}

Make the following observations in the inspect output

• State indicates the volume is attached and shows the node on which it is attached. This is the node where the Kubernetes pod is running.
• HA shows the number of configured replcas for this volume
• Labels show the name of the PVC for this volume
• Replica sets on nodes shows the px nodes on which volume is replicated

Now that we have mysqlQL up, let's proceed to creating a sample database!

In this step, we will initialize a sample database in our MySQL instance.

Step: Open a shell inside the MySQL container

Below commands exec into the MySQL pod:

POD=`kubectl get pods -l app=mysql | grep Running | grep 1/1 | awk '{print $1}'`
kubectl exec -it $POD -- bash

Next we can launch the mysql utility and show existing databases, including pxdemo which we defined in our spec.

mysql --password=$MYSQL_ROOT_PASSWORD
show databases;
exit

Step: Run mysqlslap to create some tables

MySQL has a diagnostic program, msyqlslap designed to emulate client load for a MySQL server and to report the timing of each stage. We're going to use it to create some tables and do a quick performance test.

mysqlslap --create-schema=pxdb --create="CREATE TABLE a (b int)" \
--query="INSERT INTO a VALUES (RAND())" \
--concurrency=1000 --iterations=1 --password=mysql123 --no-drop

Take a look at the tables and exit out of the container.

mysql --password=$MYSQL_ROOT_PASSWORD
show databases;
use pxdb;
show tables;
select count(*) from a;
exit
exit

In this step, we will simulate failure by cordoning the node where MySQL is running and then deleting the MySQL pod. The pod will then be resheduled by the STorage ORchestrator for Kubernetes (STORK) to make sure it lands on one of the nodes that has of replica of the data.

Step: Simulate a node failure to force MySQL to restart

First we will cordon the node where MySQL is running to simulate a node failure or network partition:

NODE=`kubectl get pods -l app=mysql -o wide | grep -v NAME | awk '{print $7}'`
kubectl cordon ${NODE}

Then delete the MySQL pod:

POD=`kubectl get pods -l app=mysql -o wide | grep -v NAME | awk '{print $1}'`
kubectl delete pod ${POD}

Once the mysql pod gets deleted, Kubernetes will start to create a new mysql pod on another node.

Step: Verify replacement pod starts running

Below commands wait till the new mysql pod is ready.

watch kubectl get pods -l app=mysql -o wide

When the pod come back up it will be in the Running state. When it does hit clear to ctrl-c and clear the screen.

Before you proceed you should uncordon your node:

kubectl uncordon ${NODE}

Now that we have the new mysql pod running, let's check if the database we previously created is still intact.

In this step, we will check the state of our sample MySQL database.

Step: Verify data is still available

Below commands exec into the running mysql pod and dump our sample database.

First find the pod name and exec into the container.

POD=`kubectl get pods -l app=mysql | grep Running | grep 1/1 | awk '{print $1}'`
kubectl exec -it $POD -- bash

Now use mysql to make sure our data is still there.

mysql --password=$MYSQL_ROOT_PASSWORD
show databases;
use pxdb;
show tables;
select count(*) from a;
exit
exit

Observe that the database table is still there and all the content intact!

In this step, we will run a benchmark and run out of space to show how easy it is to add space to a volume dynamically.

Step: Inspect the volume

You can run pxctl commands to inspect your volume:

VOL=`kubectl get pvc | grep px-mysql-pvc | awk '{print $3}'`
PX_POD=$(kubectl get pods -l name=portworx -n kube-system -o jsonpath='{.items[0].metadata.name}')
kubectl exec -it $PX_POD -n kube-system -- /opt/pwx/bin/pxctl volume inspect ${VOL}

Step: Expand the volume

Run the volume update command with pxctl to expand your volume

kubectl exec -it $PX_POD -n kube-system -- /opt/pwx/bin/pxctl volume update ${VOL} --size=20

Inspect the volume:

kubectl exec -it $PX_POD -n kube-system -- /opt/pwx/bin/pxctl volume inspect ${VOL}

As you can see the volume is now expanded and our MySQL database didn't require restarting.

kubectl get pods

In this step, we will take a snapshot, simulate destroy our database, and recover from the snapshot

Step: Take snapshot using kubectl

First, take a look at px-snap.yaml cat px-snap.yaml. Notice it creates a new PVC that we will then use to recover from. Let's use kubectl to take the snapshot:

kubectl create -f px-snap.yaml

You can see the snapshots using the following command:

kubectl get volumesnapshot,volumesnapshotdatas

Now we're going to go ahead and do something stupid because it's Katacoda and we're here to learn.

POD=`kubectl get pods -l app=mysql | grep Running | grep 1/1 | awk '{print $1}'`
kubectl exec -it $POD -- bash
mysql --password=$MYSQL_ROOT_PASSWORD
drop database pxdb;
show databases;
exit
exit

Ok, so we deleted our database, what now? Restore your snapshot and cary on.

Step: Restore the snapshot and see your data is still there

Snapshots are just like volumes so we can go ahead and use it to start a new instance of MySQL. Note here that we're leaving the old instance to carry on with it's version of the volume and we're creating a brand new instance of MySQL with the snapshot data!

First, lets create a new pvc, cat px-snap-pvc.yaml, from the snapshot:

kubectl create -f px-snap-pvc.yaml

Now, lets look at the yaml: .

Now let's deploy a second MySQL that uses this new PVC, cat mysql-app-restore.yaml, and make sure it came up.

kubectl create -f mysql-app-restore.yaml

Step: Verify mysql pod is ready

Below commands wait till the mysql pods are in ready state.

watch kubectl get pods -o wide

When the pod is in Running state then then hit clear to ctrl-c and clear the screen.

In this step, we will check the state of our sample MySQL database.

Step: Verify data

Below commands exec into the running mysql pod and dump our sample database.

First find the pod name and exec into the container.

POD=`kubectl get pods -l app=mysql-snap | grep Running | grep 1/1 | awk '{print $1}'`
kubectl exec -it $POD -- bash

Now use mysql to make sure our data is still there.

mysql --password=$MYSQL_ROOT_PASSWORD
show databases;
use pxdb;
show tables;
select count(*) from a;
exit
exit

Observe that the database table is still there and all the content intact!