Configuration with Kubernetes and Jenkins Part 1: Secrets

In this post, I will walk through how to configure Kubernetes Secrets through your Jenkins Pipeline. I’ll assume you already know how to create a Jenkins Pipeline and how to configure helm.

Jenkins stores your secret as an encrypted credential. Your Secret can be a username/password, a simple text string, or an entire file. In this example we will demonstrate using an entire Secret file.

First create your secrets as a yaml file.

secrets:
  my.secret: surprise!!
  my.other.secret: itworks!!

The secrets file will be uploaded to Jenkins and stored encrypted on disk. Once the file is uploaded it cannot be changed, only removed and re-added.

Jenkins -> Credentials -> Global Credentials

On the Credentials page:

1. Click Add Credentials at the top left and choose your Kind. For this example, I am using a secret file.
2. Use your repo name as the id so the credential can be easily tied to your repo/build (useful in a multi-tenant Jenkins).
3. Upload your secret file.

There are multiple bindings to choose from depending on what kind of credential you added to Jenkins. I used a file so the file binding is what I have in my Jekinsfile.

stage('Deploy') {
    steps {
        withCredentials([file(credentialsId: 'example-secrets-file', variable: 'SECRET_FILE')]) {
            sh '### run your helm command'
        }
    }
}

The variable will point to the location of the yaml on disk. This variable will used to reference the file in your deployment script. In this example, we used SECRET_FILE as the reference variable name.

Assuming your file is a proper yaml doc, you can pass the entire file to helm and use it as a values file

helm upgrade --install
    --set service.image.nameTag=${DOCKER_NAMETAG}
    -f ${SECRET_FILE}
    ./chart-dir

In your helm templates, you will have a Secret spec that allows you to use the Secret File passed in from your helm command.

apiVersion: v1
kind: Secret
metadata:
  name: {{ .Chart.Name }}
type: Opaque
data:
  {{- range $key, $val := .Values.secrets }}
  {{ $key }}: {{ $val | b64enc }}
  {{- end}}

Per the Secret specifications, all secrets must be base64Encoded. We do this by iterating through the yaml doc and running a base64Encode function on the secret values in helm.

Accessing Secrets in your Pod

Secrets can be added to your deployment in two ways, env vars or file mounts. I will demostrate using file mounts which are a little more secure than env vars. See the kubernetes docs to learn how to pass secrets to your pod via env vars.

In your deployment file, add a volume mount to your container to mount secrets to your pod. The mountPath will be the location where the secrets reside in your pod.

volumeMounts:
- name: secrets
  mountPath: "/secrets"
  readOnly: true

Next, add a volume which points to your Secret using the name defined in your helm chart.

volumes:
- name: secrets
  secret:
    secretName: {{ .Chart.Name }}

This will take each secret in your Secret spec and mount them as individual files in your pod. Your application can now read these secrets direclty from the filesystem.

Spring Support

If you are using Spring, Spring Cloud Kubernetes has a SecretPropertySource that will read all mounted secrets and add them to your Spring Environment. This allows you to use @Value annotations to inject them where needed.

Add this dependency to your pom.xml file.

<dependency>
    <groupId>io.fabric8</groupId>
    <artifactId>spring-cloud-starter-kubernetes</artifactId>
    <version>0.1.6</version>
</dependency>

Add this system property to your java -jar command

-Dspring.cloud.kubernetes.secrets.paths=/secrets

Use the path where you mounted the secrets in your deployment.yaml file.

That’s it. You now have access to all of the secrets in your Spring app.