Portworx (PX) requires at least some nodes in the cluster to have dedicated storage for Portworx to use. PX will then carve out virtual volumes from these storage pools. In this example, we use a 20GB block device that exists on each node.

Step: List block devices on all nodes

Click ssh [email protected] lsblk to list the available devices on the node1.

Note the storage device vdb, which will be used by PX as one of it's raw block disks. All the nodes in this setup have the vdb device.

PX relies on an external key value database (KVDB) like etcd or consul to store it's cluster discovery and bootstrap information.

Step: Create an etcd instance

Below, we are starting etcd via a docker container.

export IP=$(hostname -I | awk '{print $1}')

docker run -d --net=host -p 4001:2379 \
   --volume=/var/lib/px-etcd:/etcd-data \
   --name etcd quay.io/coreos/etcd /usr/local/bin/etcd \
   --data-dir=/etcd-data --name node1 \
   --advertise-client-urls http://${IP}:4001 \
   --listen-client-urls http://${IP}:4001 \
   --initial-advertise-peer-urls http://${IP}:2380 \
   --listen-peer-urls http://${IP}:2380 \
   --initial-cluster node1=http://${IP}:2380

For production setups, follow this page for recommendations on etcd.

Step: Verify that etcd is functional and reachable

Below command queries the etcd version endpoint from node01.

Click: ssh [email protected] curl -s http://master:4001/version

PX can be deployed with a single command in Kubernetes as a DaemonSet.

Step: Fetch the Portworx spec

VER=`kubectl version --short | awk -Fv '/Server Version: /{print $3}'`
curl -L -s -o px-spec.yaml "http://install.portworx.com?c=px-demo&k=etcd://master:4001&kbVer=${VER}&s=/dev/vdb"

Observe how we used query parameters in the curl command to customize our spec.

In our comamnd:

• c=px-demo specifies the cluster name
• k=etcd://master:4001 specifies the etcd
• kbVer=${VER} specifies the Kubernetes version
• s=/dev/vdb specifies the block device to use

Step: Apply the Portworx spec

kubectl apply -f px-spec.yaml

Step: Wait till Portworx pods are ready

kubectl get pods -n kube-system -l name=portworx -o wide

while true; do
    NUM_READY=`kubectl get pods -n kube-system -l name=portworx -o wide | grep Running | grep 1/1 | wc -l`
    if [ "${NUM_READY}" == "3" ]; then
        echo "All portworx nodes are ready !"
        kubectl get pods -n kube-system -l name=portworx -o wide
        break
    else
        echo "Waiting for portworx nodes to be ready. Current ready nodes: ${NUM_READY}"
    fi
    sleep 5
done

It can take a few seconds for Portworx to complete initialization

Step: Fetch Portworx cluster status

Portworx ships with a pxctl command line that you can use for managing Portworx.

Below command executes the pxctl status command in one of the Portworx pods to fetch the overall cluster status.

PX_POD=$(kubectl get pods -l name=portworx -n kube-system -o jsonpath='{.items[0].metadata.name}')

watch --color kubectl exec -it $PX_POD -n kube-system -- /opt/pwx/bin/pxctl status

We now have a 3-node Portworx cluster up !

Let's dive into our cluster status.

• All 3 nodes are online and use Kubernetes node names as the Portworx node IDs.- Observe that Portworx clustered the 20GB block device from each node in a 60GB storage cluster.
• Portworx detected the block device media type as magnetic and created a storage pool for this. If you have different types of devices, for example SSD, a different storage pool is created for the SSD type.