Setting Up HashiCorp Vault on Ubuntu: A Beginner's Guide
HashiCorp Vault is an open-source tool designed to manage and control access to sensitive information such as API keys, passwords, certificates, and other secrets. It’s an essential component in a secure infrastructure, enabling organizations to store and tightly control access to tokens, passwords, certificates, and encryption keys for protecting secrets and other sensitive data.
This guide covers the installation and configuration of Vault on an Ubuntu system, along with examples of how to use Vault to manage secrets. Whether you are a developer, system administrator, or DevOps engineer, this article will help you get started with Vault and integrate it into your workflow.
Table of Contents
Installation PrerequisitesInstalling Vault on Ubuntu
Configuring Vault
Managing Secrets with Vault
Working with policies
Integrating Vault with Applications
Best Practices for Using Vault
Securing Vault with TLS
Troubleshooting Common Issues
Troubleshooting with "Permission denied"
Troubleshooting with "ERROR storage migration..."
Troubleshooting with "No policy named: root"
Installation Prerequisites
Before you start installing Vault on your Ubuntu system, ensure you have the following prerequisites in place:
- OS Type: Linux
- Version: Ubuntu Budgie 22.04.2 LTS 64bit
- Memory: 2 GB
- Hard disk: 20 GB
- CPU: 4 Core
- Network Access: Your system should have internet access to download and install Vault from the HashiCorp repository.
- Sudo Privileges: You’ll need administrative (sudo) privileges to install and configure Vault.
While Vault can be installed on various Linux distributions, this guide focuses on Ubuntu.
Checking System Requirements
To check your Ubuntu version, use the following command:
lsb_release -a
 |
| lsb_release -a output |
To verify available memory and storage, you can use the following commands:
 |
| df -h output |
df -h
Ensure that your system meets the minimum requirements before proceeding.
Installing Vault on Ubuntu
Update Your System
Before installing any new software, it’s a good practice to update your package list and upgrade existing packages:sudo apt-get update sudo apt-get upgrade -y
-y tells apt-get to assume the answer to all prompts is yes
Install Required Dependencies
Vault requires a few dependencies, such as `wget` and `unzip`, to download and extract the Vault binary. Install these using:sudo apt-get install -y wget unzip
Download Vault
Visit the official [HashiCorp Vault releases page](https://developer.hashicorp.com/vault/install) to find the latest version of Vault. Use `wget` to download the binary, actually the latest release is 1.17.5:wget -O- https://apt.releases.hashicorp.com/gpg | sudo gpg --dearmor -o /usr/share/keyrings/hashicorp-archive-keyring.gpg
echo "deb [signed-by=/usr/share/keyrings/hashicorp-archive-keyring.gpg] https://apt.releases.hashicorp.com $(lsb_release -cs) main" | sudo tee /etc/apt/sources.list.d/hashicorp.list
sudo apt update && sudo apt install vault
Verify the Installation
To confirm that Vault was installed correctly, check its version:vault --version
 |
| vault --version output |
The output should display the version of Vault you installed, indicating that Vault is ready to be used.
Set the needed ownership to vault data directory
sudo chown -R vault:vault /opt/vault/data
sudo chmod -R 750 /opt/vault/data
Setting Up Vault as a Systemd Service (Optional)
To ensure that Vault starts automatically on system boot and runs as a background service, you can set it up as a systemd service. Create a systemd service file:sudo nano /etc/systemd/system/vault.service
Add the following configuration:
[Unit]
Description=HashiCorp Vault
Documentation=https://www.vaultproject.io/docs/
Requires=network-online.target
After=network-online.target
[Service]
User=vault
Group=vault
ExecStart=vault server -config=/etc/vault.d/vault.hcl
ExecReload=/bin/kill --signal HUP $MAINPID
KillMode=process
Restart=on-failure
LimitNOFILE=65536
LimitMEMLOCK=infinity
[Install]
WantedBy=multi-user.target
Reload the systemd configuration and start the Vault service:
sudo systemctl daemon-reload
sudo systemctl start vault
sudo systemctl enable vault
Configuring Vault
Setting Up a Vault Configuration File
Vault requires a configuration file to define its storage backend, listener, and other settings. Create this configuration file at `/etc/vault.d/vault.hcl` if not already present:storage "file" {
path = "/opt/vault/data"
}
listener "tcp" {
address = "127.0.0.1:8200"
tls_disable = 1
}
log_level = "debug"
ui = true
Starting Vault
Start the Vault server with the configuration you’ve just created:sudo vault server -config=/etc/vault.d/vault.hcl
or start the vault service:
sudo systemctl start vault
Vault will start in the foreground. To run it in the background, use systemd as shown in the installation step.
Initializing Vault
Vault must be initialized before it can be used. Initialization generates the master key, root token, and unseal keys. Run the following command:open a command line window and type:
vault operator init
The output will provide the unseal keys and the initial root token. Store these securely, as they are required to access Vault.
 |
| The output will provide the unseal keys and the initial root token |
Unsealing Vault
Vault starts in a sealed state, meaning it is encrypted and inaccessible. To make it operational, you need to provide a quorum of unseal keys. By default, Vault requires three of the five unseal keys generated during initialization:vault operator unseal <unseal_key_1>
vault operator unseal <unseal_key_2>
vault operator unseal <unseal_key_3>
 |
| Unsealing Vault state |
After unsealing, Vault becomes operational and can be used to store and retrieve secrets.
Configuring Authentication Methods
Vault supports various authentication methods, such as AppRole, LDAP, and GitHub. These methods allow you to control who can access Vault and what they can do. For example, to enable the `userpass` authentication method:export $(dbus-launch)
The command export $(dbus-launch) is used to start a new D-Bus session bus and set environment variables to connect to this bus. D-Bus (Desktop Bus) is a messaging system that allows applications to communicate with each other.
vault login <root-token>
Set grants permissions for all authentication paths under "auth/*" with this configuration with the capabilities to create, update, read, delete, and list.
path "auth/*" {
capabilities = ["create", "update", "read", "delete", "list"]
}
vault policy write auth-policy auth-policy.hcl
vault auth enable userpass
The command `vault policy write auth-policy auth-policy.hcl` is used to create or update a policy in HashiCorp Vault.
After enabling, you can create users and assign them policies:
vault write auth/userpass/users/username1 password="test" policies="default"
test the login method just created:
vault login -method=userpass username=username1 password=test
 |
| vault login output |
Managing Secrets with Vault
Storing and Retrieving Secrets
Vault makes it easy to store and retrieve secrets. For example, to store a secret:vault kv put secret/myapp/config username="admin" password="P@ssw0rd"
To retrieve the secret:
vault kv get secret/myapp/config
Dynamic secret
Dynamic secrets are generated on-demand and automatically expire after a certain period, minimizing the risk of misuse. For example, you can configure Vault to generate database credentials dynamically:vault secrets enable database
Set up a database configuration:
vault write database/config/my-mysql-database \
plugin_name=mysql-database-plugin \
connection_url="{{username}}:{{password}}@tcp(127.0.0.1:3306)/" \
allowed_roles="my-role" \
username="root" \
password="rootpassword"
Create a role for dynamic credentials:
vault write database/roles/my-role \
db_name=my-mysql-database \
creation_statements="CREATE USER '{{name}}'@'%' IDENTIFIED BY '{{password}}'; GRANT SELECT ON *.* TO '{{name}}'@'%';" \
default_ttl="1h" \
max_ttl="24h"
Setting up Secret Engines
Vault supports various secret engines like KV, database, and AWS. These engines are responsible for managing different types of secrets. Enable and configure the appropriate engine for your use case:vault secrets enable kv
For advanced setups, such as managing AWS IAM roles dynamically, you would enable and configure the AWS secrets engine:
vault secrets enable aws
vault write aws/config/root access_key=<AWS_ACCESS_KEY> secret_key=<AWS_SECRET_KEY> region=<AWS_REGION>
Working with policies
To view the policies assigned to a user in HashiCorp Vault, you can inspect the token of the user, as the policies are associated with the token generated when the user authenticates.Here are the steps to see the policies assigned to a user:
Use the `vault token lookup` command
The `vault token lookup` command allows you to view the details of a token, including the list of policies associated with that token.If you have the user's token, you can run:
vault token lookup <token>
 |
| vault token lookup <token> output |
This command will display various information about the token, including the policies section, which lists the active policies for that user. For example, the output will include something like this:
policies [default]
Use the `token_accessor`
If you don't have the full token but only the `token_accessor`, you can use:vault token lookup -accessor <token_accessor>
This command will return the same information as using the full token, including the list of policies associated with the token.
Check policies via authentication
If the user authenticates through an authentication method like userpass, LDAP, or others, policies are assigned at the time of authentication. You can check the policies configured for that authentication method by inspecting the configuration.For example, for the `userpass` method, you can verify the policies assigned to a specific user:
vault read auth/userpass/users/<username>
as an example:
vault read auth/userpass/users/username1
 |
| The output will show the policies associated with that user. |
The output will show the policies associated with that user.
Inspect policies for an entity or group (optional)
If you're using Vault's Identity system (which allows managing users and groups), you can also assign policies to an entity or group. You can run the following command to see the policies assigned to an entity:vault read identity/entity/id/<entity_id>
as an example:
vault read identity/entity/id/e2fa72d2-29fe-745d-946f-4f7c87f486cf
 |
| vault read output |
Or for groups:
vault read identity/group/id/<group_id>
Check Policy Capabilities for a Specific Path
If you want to see what specific capabilities a policy has for a given path (e.g., whether it can read, write, or list secrets at a certain path), you can use the `vault capabilities` command.For example, to check what capabilities a token has for a specific path:
vault token capabilities <token> <path>
Example:
vault token capabilities <token> secret/data/my-secret
This will output the capabilities for that token on the specified path:
read
create
update
delete
list
```
Integrating Vault with Applications
Integrating Vault with your applications ensures that secrets are managed securely across your infrastructure. Here’s an example of integrating Vault with a web application:
1. Install Vault Agent: Install Vault Agent on your application server. The agent will handle authentication and secret retrieval.
2. Configure Vault Agent: Create a configuration file for the Vault Agent to authenticate with Vault and retrieve secrets:
vault {
address = "http://127.0.0.1:8200"
}
auto_auth {
method "approle" {
mount_path = "auth/approle"
config = {
role_id_file_path = "/etc/vault/role_id"
secret_id_file_path = "/etc/vault/secret_id"
}
}
sink "file" {
config = {
path = "/etc/vault/secrets.json"
}
}
}
3. Update Your Application: Modify your application to read secrets from the Vault Agent’s output file (`/etc/vault/secrets.json`). This ensures that your application always has access to the most current secrets without hardcoding them.
Best Practices for Using Vault
Secure the Unseal Keys
The unseal keys are critical for accessing Vault. Use Shamir’s Secret Sharing to distribute these keys among trusted individuals, and store them securely, such as in a hardware security module (HSM) or secure vault.Enable Audit Logging
Always enable audit logging to track who accesses Vault and what actions they perform. This is essential for compliance and forensic analysis in case of a security incident.vault audit enable file file_path=/var/log/vault_audit.log
Use Least Privilege
Apply the principle of least privilege when assigning policies. Only grant users and applications the permissions they need to perform their tasks. Review and update policies regularly to ensure they are up-to-date.Regularly Rotate Secrets
Secrets should be rotated regularly to minimize the risk of them being compromised. Vault supports automatic secret rotation for certain secret types, such as database credentials and SSH keys.Monitor Vault Health
Use monitoring tools to keep an eye on Vault’s performance and health. HashiCorp provides integrations with popular monitoring systems like Prometheus and Grafana, allowing you to set up alerts and dashboards to monitor Vault.Backup Vault Data
Regularly back up Vault’s data to ensure you can recover it in case of data loss. Use Vault’s built-in snapshot functionality or a third-party backup solution.Securing Vault with TLS
Configuring TLS for Vault
Transport Layer Security (TLS) is crucial for securing communication between Vault clients and the Vault server. Configure Vault to use TLS by generating or acquiring a certificate and setting the appropriate options in the configuration file.Update the `vault.hcl` configuration:
listener "tcp" {
address = "127.0.0.1:8200"
tls_cert_file = "/etc/vault.d/vault.crt"
tls_key_file = "/etc/vault.d/vault.key"
}
Troubleshooting Common Issues
Vault Won’t Start
If Vault fails to start, check the configuration file for syntax errors and ensure that all paths specified in the configuration exist and are accessible.Unseal Keys Are Lost
If you lose your unseal keys, you cannot access Vault. It’s crucial to back up these keys securely. In the event of lost keys, you must rely on your backup data and reinitialize Vault.Authentication Issues
Ensure that the correct authentication method is enabled and properly configured. If users or applications cannot authenticate, check the policies assigned to ensure they have the necessary permissions.Performance Issues
If you experience performance issues, consider scaling your Vault deployment, enabling caching, or optimizing your storage backend. Monitoring tools can help identify bottlenecks.Troubleshooting with "Permission denied"
The "permission denied" error when running vault command such as `vault token lookup <token>` indicates that the token you're using does not have sufficient permissions to perform the lookup operation on another token or even on its own token, depending on the scenario.To resolve this, here’s what you can do:
Ensure You Have the Right Policies
To look up a token, your token needs to have the necessary permissions on the system path that controls token management: `sys/token`.For the `vault token lookup` command to work, the policy associated with your token needs to include access to the following path:
path "auth/token/lookup-self" {
capabilities = ["read"]
}
If you want to look up other tokens (not just your own), you need access to:
path "auth/token/lookup" {
capabilities = ["read"]
}
You can create a policy with these capabilities, and attach it to the token being used.
vault policy write token-lookup token-lookup.hcl
Assign this policy to your token or entity, if you are using a specific token:
vault token create -policy="token-lookup"
Log in again using the token with this policy and attempt the lookup command.
Look Up Your Own Token
If you want to look up your own token, you can use the `vault token lookup` command without specifying the token explicitly. This will attempt to look up the token that is currently in use.Vault token lookup
If you only have access to view your own token details, this will return the information about the token you're using without needing additional permissions.Use a Token with More Permissions (Root Token or Admin Token)
If you are trying to look up tokens other than your own and don’t have the necessary permissions, you can:Log in with a token that has broader permissions, such as a root token or another admin token that has been given access to the `auth/token/lookup` path.
If you have access to the root token, you can use it to bypass most permission restrictions. To log in with the root token, use:
vault login <root-token>
Then, you should be able to run the `vault token lookup <token>` command successfully.
Check Vault Status
If your Vault instance is in a sealed or standby state, some operations might be restricted. You can check the status of Vault with:vault status
Ensure Vault is in an unsealed and active state.
Troubleshooting with "ERROR storage migration..."
ERROR storage migration check error: error="open /opt/vault/data/core/_migration: permission denied"The error message indicates that Vault does not have the necessary permissions to access the `/opt/vault/data/core/_migration` file or directory during a storage migration. Here’s how to resolve this issue:
Check Permissions for the Vault Data Directory
Ensure that the user running the Vault service has the necessary permissions to access the directory:sudo chown -R vault:vault /opt/vault/data
sudo chmod -R 750 /opt/vault/data
This sets the owner to `vault` and grants the owner read, write, and execute permissions, while the group has read and execute permissions.
Verify the User Running Vault
Check which user is configured to run the Vault service. You can find this in the Vault systemd service file (often located at `/etc/systemd/system/vault.service`):sudo cat /etc/systemd/system/vault.service | grep User
Ensure that the user specified here matches the user that owns the `/opt/vault/data` directory.
Restart the Vault Service
After adjusting the permissions, restart the Vault service to apply the changes:sudo systemctl restart vault
Check the Logs
If the issue persists, check the Vault logs for more details:journalctl -u vault -f
This should help you confirm if there are any remaining permission issues or other errors.
Troubleshooting with "No policy named: root"
The "No policy named: root" message you received when trying to read the `root` policy with `vault policy read root` is expected because the `root` token in HashiCorp Vault is a special, built-in token that is not tied to a regular policy. The `root` token has full, unrestricted access to Vault and does not follow the standard policy system.
Here’s why you’re seeing this and how to manage policies in Vault:
Why You Can't See the `root` Policy:
The `root` token is not based on any policy: The `root` token is a special, built-in token that grants complete access to everything in Vault. It's not created or managed like other tokens with policies attached. As a result, Vault doesn't have a `root` policy that can be read with `vault policy read root`.
Root is a built-in token with full privileges: Since the root token has full control over Vault (including all system paths and operations), there's no need for a defined `root` policy. Instead, the root token bypasses all policy checks.
What You Can Do Instead:
1. Check or Read Other Policies
If you want to view policies that are associated with other tokens or users, you can use the command:
vault policy read <policy_name>
For example, to read the `default` policy, you can run:
vault policy read default
If you've created custom policies, you can replace `<policy_name>` with the name of your custom policy.
2. Manage Custom Policies:
You can create, update, and read custom policies in Vault to control access to various paths. For example, if you want to create a new policy, you can write it in a file, then apply it to Vault:
path "secret/*" {
capabilities = ["create", "read", "update", "delete", "list"]
}
Save this to a file called `my-policy.hcl` and then apply it:
vault policy write my-policy my-policy.hcl
Afterward, you can view this policy with:
vault policy read my-policy
3. Use the Root Token Wisely:
Since the `root` token has unlimited power in Vault, it’s usually best to avoid using it for day-to-day operations. Instead, create specific policies that grant the necessary permissions for users or services and use tokens that have those policies attached.
If you ever lose access to a root token, you can generate a new one by unsealing the Vault and following the recovery process if needed.
Conclusion
In this guide, we've explored the fundamentals of installing and configuring HashiCorp Vault on Ubuntu. While understanding Vault in isolation is crucial, its true value emerges when integrated into broader, dynamic systems. In an upcoming article, we will delve into the interaction between application environments, Vault agents, and backend databases, showcasing how secrets management operates in real-world scenarios.References
- HashiCorp Vault Official Documentation
- Vault Secret Engines
- Vault Authentication Methods
- Introduction to HashiCorp Vault: A Comprehensive Guide and Essential Commands
- Illustrations generated with the help of OpenAI's AI.
About Me
I am passionate about IT technologies. If you’re interested in learning more or staying updated with my latest articles, feel free to connect with me on:Personal Website
Feel free to reach out through any of these platforms if you have any questions!
I am excited to hear your thoughts! 👇
Comments
Post a Comment