Difficulty: beginner
Estimated Time: 5 minutes

Logo

If you are new to Vault, please complete the Vault Operations first, and then proceed with this Katacoda scenario to better understand the workflow.


Vault Deployment Guide recommended Vault to use Consul as its storage backend. The challenge is that when Vault encounters an outage, the root cause may be the storage backend. Therefore, you had to troubleshoot two different system to bring Vault to a healthy state.

In Vault 1.2, an integrated storage was introduced to persist the encrypted data. The Raft algorithm used in Consul is directly embedded into Vault to provide a built-in storage.

NOTE: Vault Integrated Storage is currently in Technical Preview mode; therefore, not suitable for deployment in production.

[TECH PREVIEW] Vault Integrated Storage

Step 1 of 4

Scenario Introduction


Wait until the initial setup completes before start.

In this tutorial, you are going to create a highly available (HA) Vault cluster using the integrated storage backend as its persistent storage.

For the purpose of demonstration, you are going to run 3 Vault server instances each listens to a different port: node1 listens to port 8200, node2 listens to port 2200 and node3 listens to port 3200.

Start Vault Server 1 (node1)

First review the server configuration file, config-node1.hcl.

storage "raft" {
  path    = "/home/scrapbook/tutorial/raft-node1/"
  node_id = "node1"
}

listener "tcp" {
  address = "127.0.0.1:8200"
  cluster_address = "127.0.0.1:8201"
  tls_disable = true
}

disable_mlock = true
api_addr = "http://127.0.0.1:8200"
cluster_addr = "http://127.0.0.1:8201"

The storage stanza is set to use raft which is the integrated storage. The path specifies the filesystem path where the data gets stored. The node_id sets the identifier for this node in the cluster. In this case, the node ID is node1.

Enter the following command to start the node1 Vault server.

Click on the command () will automatically copy it into the terminal and execute it.

mkdir raft-node1
vault server -config=config-node1.hcl

Scroll up the Terminal to locate the following output:

==> Vault server configuration:

             Api Address: http://127.0.0.1:8200
                     Cgo: disabled
         Cluster Address: https://127.0.0.1:8201
              Listener 1: tcp (addr: "127.0.0.1:8200", cluster address: "127.0.0.1:8201", max_request_duration: "1m30s", max_request_size: "33554432", tls: "disabled")
               Log Level: info
                   Mlock: supported: true, enabled: false
                 Storage: raft (HA available)
                 Version: Vault v1.2.2

==> Vault server started! Log data will stream in below:

Now, you need to initialize and unseal the Vault server (node1).


Initialize and Unseal node1

Click the + next to the opened Terminal, and select Open New Terminal.

New Terminal

In the Terminal 2, set the VAULT_ADDR environment variable:

export VAULT_ADDR='http://127.0.0.1:8200'

Now, execute the vault operator init command to initialize the node1:

vault operator init -key-shares=1 -key-threshold=1 > key.txt

NOTE: For the simplicity, setting the number of unseal keys to 1 as well as the key threshold, and storing the generated unseal key and initial root token in a local file named, key.txt.

Execute the vault operator unseal command to enter unseal node1:

vault operator unseal $(grep 'Key 1:' key.txt | awk '{print $NF}')
Key                    Value
---                    -----
Seal Type              shamir
Initialized            true
Sealed                 false
Total Shares           1
Threshold              1
...

Return to the first Terminal and examine the server log.

Notice that right after node1 was unsealed, it first goes into standby mode.

...
[INFO]  core: vault is unsealed
[INFO]  core: entering standby mode
[INFO]  storage.raft: Node at 127.0.0.1:8201 [Follower] entering Follower state (Leader: "")
...

Since node1 is currently the only cluster member, it gets elected to be the leader.

...
[WARN]  storage.raft: Heartbeat timeout from "" reached, starting election
[INFO]  storage.raft: Node at 127.0.0.1:8201 [Candidate] entering Candidate state in term 2
[INFO]  storage.raft: Election won. Tally: 1
[INFO]  storage.raft: Node at 127.0.0.1:8201 [Leader] entering Leader state
[INFO]  core: acquired lock, enabling active operation
[INFO]  core: post-unseal setup starting
...
[INFO]  core: post-unseal setup complete

Now, node1 is ready!


Log into Vault using the initial root token (key.txt):

vault login $(grep 'Initial Root Token:' key.txt | awk '{print $NF}')

Execute the following command to view the node1's Raft cluster configuration.

vault operator raft configuration -format=json
{
  ...
  "data": {
    "config": {
      "index": 1,
      "servers": [
        {
          "address": "127.0.0.1:8201",
          "leader": true,
          "node_id": "node1",
          "protocol_version": "3",
          "voter": true
        }
      ]
    }
  },
  "warnings": null
}

At this point, node1 is the only cluster member; therefore, it becomes the leader by default.