Agenda

Agenda

(1) Intro

1. About Me
2. Objectives

Agenda

(1) Intro

1. About Me
2. Objectives

(2) Theory

1. What is Vault?
2. Why Vault?
3. Using Vault

Agenda

(1) Intro

1. About Me
2. Objectives

(2) Theory

1. What is Vault?
2. Why Vault?
3. Using Vault

(3) Practice

1. Project Overview
2. Storage Backends
3. Init, Unseal, and Authenticate
4. Auditing
5. Static Secrets
6. Dynamic Secrets
7. Encryption as a Service

Agenda

(1) Intro

1. About Me
2. Objectives

(2) Theory

1. What is Vault?
2. Why Vault?
3. Using Vault

(3) Practice

1. Project Overview
2. Storage Backends
3. Init, Unseal, and Authenticate
4. Auditing
5. Static Secrets
6. Dynamic Secrets
7. Encryption as a Service

(4) Next Steps

About Michael

$ whoami
michael.herman

About Michael

$ whoami
michael.herman

Day Job

Software Engineer at ClickFox.  

About Michael

$ whoami
michael.herman

Day Job

Software Engineer at ClickFox.  

Also

1. Co-founder/author of Real Python
2. 😍 - tech writing/education, open source, financial models, radiohead

Objectives

By the end of this talk, you should be able to...

Objectives

By the end of this talk, you should be able to...

1. Explain what Vault is and why you may want to use it

Objectives

By the end of this talk, you should be able to...

1. Explain what Vault is and why you may want to use it

2. Describe the basic Vault architecture along with dynamic and static secrets, the various backends (storage, secret, auth, audit), and how Vault can be used as an "encryption as a service"

Objectives

By the end of this talk, you should be able to...

1. Explain what Vault is and why you may want to use it

2. Describe the basic Vault architecture along with dynamic and static secrets, the various backends (storage, secret, auth, audit), and how Vault can be used as an "encryption as a service"

3. Spin up Vault with the Filesystem backend

Objectives

By the end of this talk, you should be able to...

1. Explain what Vault is and why you may want to use it

2. Describe the basic Vault architecture along with dynamic and static secrets, the various backends (storage, secret, auth, audit), and how Vault can be used as an "encryption as a service"

3. Spin up Vault with the Filesystem backend

4. Init and unseal Vault

Objectives

By the end of this talk, you should be able to...

1. Explain what Vault is and why you may want to use it

2. Describe the basic Vault architecture along with dynamic and static secrets, the various backends (storage, secret, auth, audit), and how Vault can be used as an "encryption as a service"

3. Spin up Vault with the Filesystem backend

4. Init and unseal Vault

5. Authenticate against Vault

Objectives

By the end of this talk, you should be able to...

1. Explain what Vault is and why you may want to use it

2. Describe the basic Vault architecture along with dynamic and static secrets, the various backends (storage, secret, auth, audit), and how Vault can be used as an "encryption as a service"

3. Spin up Vault with the Filesystem backend

4. Init and unseal Vault

5. Authenticate against Vault

6. Configure an Audit backend

Objectives

By the end of this talk, you should be able to...

1. Explain what Vault is and why you may want to use it

2. Describe the basic Vault architecture along with dynamic and static secrets, the various backends (storage, secret, auth, audit), and how Vault can be used as an "encryption as a service"

3. Spin up Vault with the Filesystem backend

4. Init and unseal Vault

5. Authenticate against Vault

6. Configure an Audit backend

7. Work with static and dynamic secrets

Objectives

By the end of this talk, you should be able to...

1. Explain what Vault is and why you may want to use it

2. Describe the basic Vault architecture along with dynamic and static secrets, the various backends (storage, secret, auth, audit), and how Vault can be used as an "encryption as a service"

3. Spin up Vault with the Filesystem backend

4. Init and unseal Vault

5. Authenticate against Vault

6. Configure an Audit backend

7. Work with static and dynamic secrets

8. Use the Transit backend as an "encryption as a service"

What is Vault?

What is Vault?

Vault is an open-source tool used for securely storing and managing secrets.

What is Vault?

Vault is an open-source tool used for securely storing and managing secrets.

Secrets?

What is Vault?

Vault is an open-source tool used for securely storing and managing secrets.

Secrets?

Secrets are securely-sensitive or personally identifiable info like database credentials, SSH keys, usernames/passwords, AWS IAM credentials, API tokens, SSNs, credit card numbers.

What is Vault?

Vault is an open-source tool used for securely storing and managing secrets.

Secrets?

Secrets are securely-sensitive or personally identifiable info like database credentials, SSH keys, usernames/passwords, AWS IAM credentials, API tokens, SSNs, credit card numbers.

Distributing secrets

1. Who has access to them?
2. Who manages them?
3. How do you control who has access to them?
4. How do your apps get them?
5. How are they updated?
6. How are they revoked?

Why Vault?

Why Vault?

Vault provides answers to the questions on the previous slide and helps to solve the following problems with secret management:

https://www.vaultproject.io/intro/index.html

Using Vault (1)

Using Vault (1)

Secret Management

Vault has two types of secrets:

1. Static secrets (think encrypted Redis or Memcached) have refresh intervals but they do not expire unless explicitly revoked. They are defined ahead of time and then shared.

   

2. Dynamic secrets have enforced leases and generally expire after a short period of time. Since they don’t exist until they are accessed, there’s less exposure - so dynamic secrets are much more secure.

Using Vault (1)

Secret Management

Vault has two types of secrets:

1. Static secrets (think encrypted Redis or Memcached) have refresh intervals but they do not expire unless explicitly revoked. They are defined ahead of time and then shared.

   

2. Dynamic secrets have enforced leases and generally expire after a short period of time. Since they don’t exist until they are accessed, there’s less exposure - so dynamic secrets are much more secure.

Encryption as a Service

The Transit backend can be used as an "encryption as a service":

1. Encrypt and decrypt data "in-transit" without storing it
2. Easily integrate encryption into your application workflow

Using Vault (2)

Using Vault (2)

Backends are everywhere!

* used in this presentation

Using Vault (2)

Backends are everywhere!

* used in this presentation

vault-architecture.png

https://www.vaultproject.io/docs/internals/architecture.html

Project Overview

https://github.com/testdrivenio/vault-consul-docker

├── docker-compose.yml
└─ vault
    ├── Dockerfile
    ├── config
    │   └── vault-config.json
    ├── data
    ├── logs
    └── policies
        └── app-policy.json

Project Overview

https://github.com/testdrivenio/vault-consul-docker

├── docker-compose.yml
└─ vault
    ├── Dockerfile
    ├── config
    │   └── vault-config.json
    ├── data
    ├── logs
    └── policies
        └── app-policy.json

Build the image and run the container:

$ docker-compose up -d --build

Project Overview

https://github.com/testdrivenio/vault-consul-docker

├── docker-compose.yml
└─ vault
    ├── Dockerfile
    ├── config
    │   └── vault-config.json
    ├── data
    ├── logs
    └── policies
        └── app-policy.json

Build the image and run the container:

$ docker-compose up -d --build

Start a bash session within the running container:

$ docker-compose exec vault bash

Storage Backends

Storage Backends

Vault works with a number of storage backends that are used for storing the secrets - in-memory, files, various SQL and NoSQL databases, Consul, S3, etc.

https://www.vaultproject.io/docs/configuration/storage/index.html

Storage Backends

Vault works with a number of storage backends that are used for storing the secrets - in-memory, files, various SQL and NoSQL databases, Consul, S3, etc.

https://www.vaultproject.io/docs/configuration/storage/index.html

To keep things simple, the examples in this presentation use the Filesystem backend:

{
  "backend": {
    "file": {
      "path": "vault/data"
    }
  },
  "listener": {
    "tcp":{
      "address": "0.0.0.0:8200",
      "tls_disable": 1
    }
  },
  "ui": true
}

Init, Unseal, and Authenticate

Init, Unseal, and Authenticate

Initialize Vault:

$ vault operator init

Take note of the unseal keys and the initial root token. You will need to provide 3 of the unseal keys every time the Vault server is re-sealed.

(Why 3 keys? https://en.wikipedia.org/wiki/Shamir's_Secret_Sharing)

Init, Unseal, and Authenticate

Initialize Vault:

$ vault operator init

Take note of the unseal keys and the initial root token. You will need to provide 3 of the unseal keys every time the Vault server is re-sealed.

(Why 3 keys? https://en.wikipedia.org/wiki/Shamir's_Secret_Sharing)

Now you can unseal the Vault using 3 of the keys:

$ vault operator unseal

Authenticate:

$ vault login
Token (will be hidden):

This uses the root policy. In production you'll want to set up policies with different levels of access.

https://www.vaultproject.io/docs/concepts/policies.html

Auditing

Auditing

Before we test out the functionality, let’s enable an Audit Device:

$ vault audit enable file file_path=/vault/logs/audit.log
Success! Enabled the file audit device at: file/

Auditing

Before we test out the functionality, let’s enable an Audit Device:

$ vault audit enable file file_path=/vault/logs/audit.log
Success! Enabled the file audit device at: file/

To test, run the following command to view all enabled Audit Devices:

$ vault audit list
Path     Type    Description
----     ----    -----------
file/    file    n/a

This request should be logged in vault/logs/audit.log.

Static Secrets

Static Secrets

Vault can be managed through the CLI, HTTP API, or UI.

Static Secrets

Vault can be managed through the CLI, HTTP API, or UI.

CLI Examples

https://www.vaultproject.io/docs/secrets/kv/index.html

Create:

$ vault kv put secret/foo bar=precious
Success! Data written to: secret/foo

Read:

$ vault kv get secret/foo

Delete:

$ vault kv delete secret/foo

Static Secrets

Vault can be managed through the CLI, HTTP API, or UI.

CLI Examples

https://www.vaultproject.io/docs/secrets/kv/index.html

Create:

$ vault kv put secret/foo bar=precious
Success! Data written to: secret/foo

Read:

$ vault kv get secret/foo

Delete:

$ vault kv delete secret/foo

Take note of the audit log. Each of the above requests were logged!

Dynamic Secrets (1)

Dynamic Secrets (1)

Vault supports a number of dynamic secret backends for generating secrets dynamically when needed.

For example, with the AWS and Google Cloud backends, you can create access credentials based on IAM policies. The databases backend, meanwhile, generates database credentials based on configured roles.

Dynamic Secrets (1)

Vault supports a number of dynamic secret backends for generating secrets dynamically when needed.

For example, with the AWS and Google Cloud backends, you can create access credentials based on IAM policies. The databases backend, meanwhile, generates database credentials based on configured roles.

Dynamic Secrets:

• are generated on demand
• have limited access based on role
• are "leased"
• can be revoked
• come with an audit trail

Dynamic Secrets (2)

Dynamic Secrets (2)

Generate dynamic, on-demand AWS access credentials

Enable:

$ vault secrets enable -path=aws aws
Success! Enabled the aws secrets engine at: aws/

Authenticate:

$ vault write aws/config/root access_key=foo secret_key=bar
Success! Data written to: aws/config/root

Create role and credentials:

$ vault write aws/roles/ec2-read \
  arn=arn:aws:iam::aws:policy/AmazonEC2ReadOnlyAccess
$ vault read aws/creds/ec2-read

Dynamic Secrets (2)

Generate dynamic, on-demand AWS access credentials

Enable:

$ vault secrets enable -path=aws aws
Success! Enabled the aws secrets engine at: aws/

Authenticate:

$ vault write aws/config/root access_key=foo secret_key=bar
Success! Data written to: aws/config/root

Create role and credentials:

$ vault write aws/roles/ec2-read \
  arn=arn:aws:iam::aws:policy/AmazonEC2ReadOnlyAccess
$ vault read aws/creds/ec2-read

Secrets are generated when they are requested (i.e., a web app requests access to S3). They are not available in the store before this.

https://www.vaultproject.io/intro/getting-started/dynamic-secrets.html

Encryption as a Service

Encryption as a Service

Enable transit:

$ vault secrets enable transit
Success! Enabled the transit secrets engine at: transit/

Encryption as a Service

Enable transit:

$ vault secrets enable transit
Success! Enabled the transit secrets engine at: transit/

Configure named encryption key:

$ vault write -f transit/keys/foo
Success! Data written to: transit/keys/foo

Encryption as a Service

Enable transit:

$ vault secrets enable transit
Success! Enabled the transit secrets engine at: transit/

Configure named encryption key:

$ vault write -f transit/keys/foo
Success! Data written to: transit/keys/foo

Encrypt and decrypt:

$ vault write transit/encrypt/foo plaintext=$(base64 <<< "my precious")
Key           Value
---           -----
ciphertext    vault:v1:/Tun95IT+dVTvDfYiCHdI5rGPSAxgvPcFaDDtneRorQCyBOgv9mPKw==
$ vault write transit/decrypt/foo ciphertext=vault:v1:/Tun95IT+dVTvDfYiCHdI5rGPSAxgvPcFaDDtneRorQCyBOgv9mPKw==
Key          Value
---          -----
plaintext    bXkgcHJlY2lvdXMK
$ base64 -d <<< "bXkgcHJlY2lvdXMK"
my precious

That's it!

What's next?

That's it!

What's next?

What has not been covered?

1. Leases and revocation
2. Setting up new policies
3. Deployment
4. Configuring TLS
5. High availability

That's it!

What's next?

What has not been covered?

1. Leases and revocation
2. Setting up new policies
3. Deployment
4. Configuring TLS
5. High availability

Resources

1. Full blog post - https://testdriven.io/managing-secrets-with-vault-and-consul
2. Slides - http://mherman.org/presentations/vault
3. Repo - https://github.com/testdrivenio/vault-docker-example
4. Why We Need Dynamic Secrets - https://www.hashicorp.com/blog/why-we-need-dynamic-secrets (advantages of using dynamic secrets)

That's it!

What's next?

What has not been covered?

1. Leases and revocation
2. Setting up new policies
3. Deployment
4. Configuring TLS
5. High availability

Resources

1. Full blog post - https://testdriven.io/managing-secrets-with-vault-and-consul
2. Slides - http://mherman.org/presentations/vault
3. Repo - https://github.com/testdrivenio/vault-docker-example
4. Why We Need Dynamic Secrets - https://www.hashicorp.com/blog/why-we-need-dynamic-secrets (advantages of using dynamic secrets)

Questions?

✌️