You then use these credentials to create an intermediate CA certificate and key that can be used by Gloo Mesh Enterprise to automatically sign and issue client TLS certificates for the workload clusters.

For more information about this approach, see Bring your own CAs with automatic client TLS certificate rotation.

Single cluster

Use the steps in this guide to generate custom TLS certificates with OpenSSL.

Create the root CA credentials

Create root CA credentials that are stored in the cluster. Note that for security purposes, you might store the root CA credentials separately from the cluster. Then, you use the root CA to sign a subsequent intermediate CA certificate and key that are stored in the cluster. These intermediate CA certificate and key become the “root” CA certificate and key in the subsequent steps.

  1. Create a self-signed root CA certificate and key.
      openssl req -new -newkey rsa:4096 -x509 -sha256 \
 -days 3650 -nodes -out relay-root-ca.crt -keyout relay-root-ca.key \
 -subj "/CN=relay-root-ca" \
 -addext "keyUsage = keyCertSign"
  2. If it doesn’t already exist, create the gloo-mesh namespace.
      kubectl create namespace gloo-mesh
  3. Store the root CA certificate in the relay-root-tls-secret Kubernetes secret.
      kubectl create secret generic relay-root-tls-secret -n gloo-mesh \
 --from-file=tls.crt=relay-root-ca.crt \
 --from-file=ca.crt=relay-root-ca.crt \
 --from-file=tls.key=relay-root-ca.key

Create the intermediate CA credentials

Use the root CA key to generate an intermediate CA certificate and key. These credentials are later used to sign client TLS certificates for the Gloo agent.

  1. Create the configuration for the intermediate CA.

      cat > "relay-intermediate-ca.conf" <<EOF
[req]
req_extensions = v3_req
distinguished_name = req_distinguished_name
[req_distinguished_name]
[ v3_req ]
basicConstraints = CA:TRUE
subjectKeyIdentifier = hash
authorityKeyIdentifier = keyid:always,issuer:always
keyUsage = digitalSignature, keyEncipherment, keyCertSign
extendedKeyUsage = clientAuth, serverAuth
subjectAltName = @alt_names
[alt_names]
DNS = *.gloo-mesh
EOF

  2. Generate the private key for the intermediate CA.

      openssl genrsa -out "relay-int.key" 2048

  3. Generate the certificate signing request (CSR).

      openssl req -new -key "relay-int.key" -out relay-int.csr -subj "/CN=relay-int"

  4. Sign the CSR with the root CA key.

      openssl x509 -req \
 -days 3650 \
 -CA relay-root-ca.crt -CAkey relay-root-ca.key \
 -set_serial 0 \
 -in relay-int.csr -out relay-int.crt \
 -extensions v3_req -extfile "relay-intermediate-ca.conf"

  5. Save the intermediate CA certificate and key in the relay-tls-signing-secret Kubernetes secret.

      kubectl create secret generic relay-tls-signing-secret -n gloo-mesh \
 --from-file=tls.crt=relay-int.crt \
 --from-file=tls.key=relay-int.key \
 --from-file=ca.crt=relay-root-ca.crt

Create the server TLS certificate

Use the root CA credentials to generate a TLS certificate for the Gloo management server.

  1. Create the configuration for the server TLS certificate.
      cat >"relay-server.conf" <<EOF
[req]
req_extensions = v3_req
distinguished_name = req_distinguished_name
[req_distinguished_name]
[ v3_req ]
basicConstraints = CA:FALSE
keyUsage = nonRepudiation, digitalSignature, keyEncipherment
extendedKeyUsage = clientAuth, serverAuth
subjectAltName = @alt_names
[alt_names]
DNS = *.gloo-mesh
EOF
  2. Generate the private key for the Gloo management server.
      openssl genrsa -out "relay-server.key" 2048
  3. Generate the certificate signing request for the Gloo management server.
      openssl req -new -key "relay-server.key" -out "relay-server.csr" -subj "/CN=*.gloo-mesh" -config "relay-server.conf"
  4. Use the root CA credentials that you created earlier to sign the certificate signing request and create the server TLS certificate.
      openssl x509 -req \
 -days 3650 \
 -CA relay-root-ca.crt -CAkey relay-root-ca.key \
 -set_serial 0 \
 -in relay-server.csr -out relay-server.crt \
 -extensions v3_req -extfile "relay-server.conf"
  5. Store the server TLS certificate, private key, and root CA in the relay-server-tls-secret Kubernetes secret.
      kubectl create secret generic relay-server-tls-secret -n gloo-mesh \
 --from-file=tls.crt=relay-server.crt \
 --from-file=tls.key=relay-server.key \
 --from-file=ca.crt=relay-root-ca.crt

Create the certificate chain for the telemetry collector

  1. Store the root CA certificate in the telemetry-root-secret Kubernetes secret.
      kubectl create secret generic telemetry-root-secret \
 --from-file=ca.crt=relay-root-ca.crt \
 --namespace gloo-mesh

Create relay identity tokens

To establish initial trust between the Gloo management server and agent, you must set up your own relay identity tokens.

  1. Create an environment variable with your identity token. The token can be any string value.

      export TOKEN="<identity_token>"

  2. Store the token in the relay-identity-token-secret Kubernetes secret.

      kubectl create secret generic relay-identity-token-secret -n gloo-mesh --from-literal=token=$TOKEN

Install Gloo Network

  1. Follow the Install with Helm guide.
  2. In your Helm values file, add the following values.
      glooMgmtServer: 
  enabled: true
  registerCluster: true
  serviceType: ClusterIP
  relay:
    disableCaCertGeneration: true
    signingTlsSecret:
      name: relay-tls-signing-secret
      namespace: gloo-mesh
    tlsSecret:
      name: relay-server-tls-secret
      namespace: gloo-mesh
    tokenSecret:
      key: token
      name: relay-identity-token-secret
      namespace: gloo-mesh
glooAgent: 
  enabled: true
  relay:
    authority: gloo-mesh-mgmt-server.gloo-mesh
    serverAddress: gloo-mesh-mgmt-server.gloo-mesh:9900
    clientTlsSecretRotationGracePeriodRatio: ""
    rootTlsSecret:
      name: relay-root-tls-secret
      namespace: gloo-mesh
    tokenSecret:
      key: token
      name: relay-identity-token-secret
      namespace: gloo-mesh
telemetryCollector: 
  enabled: true
  extraVolumes: 
    - name: root-ca
      secret:
        defaultMode: 420
        optional: true
        secretName: telemetry-root-secret
    - configMap:
        items:
          - key: relay
            path: relay.yaml
        name: gloo-telemetry-collector-config
      name: telemetry-configmap
    - hostPath:
        path: /var/run/cilium
        type: DirectoryOrCreate
      name: cilium-run
    Helm valueDescription
    glooMgmtServer.relay.disableCaCertGenerationDisable the generation of self-signed certificates to secure the relay connection between the Gloo management server and agent.
    glooMgmtServer.relay.signingTlsSecretAdd the name and namespace of the Kubernetes secret that holds the intermediate CA credentials that you created earlier.
    glooMgmtServer.relay.tlsSecretAdd the name and namespace of the Kubernetes secret that holds the server TLS certificate for the Gloo management server that you created earlier.
    glooMgmtServer.relay.tokenSecretAdd the name, namespace, and key of the Kubernetes secret that holds the relay identity token that you created earlier.
    glooAgent.relay.rootTlsSecretAdd the name and namespace of the Kubernetes secret that holds the root CA credentials that you created earlier.
    glooAgent.relay.tokenSecretAdd the name, namespace, and key of the Kubernetes secret that holds the relay identity token that you created earlier.
    telemetryCollector.extraVolumesAdd the telemetry-root-secret Kubernetes secret that you created earlier to the root-ca volume. Make sure that you also add the other volumes to your telemetry collector configuration.

Multicluster

Use the steps in this guide to generate custom TLS certificates with OpenSSL.

Create the root CA credentials

Create root CA credentials that are stored in the management cluster. Note that for security purposes, you might store the root CA credentials separately from the cluster. Then, you use the root CA to sign a subsequent intermediate CA certificate and key that are stored in the cluster. These intermediate CA certificate and key become the “root” CA certificate and key in the subsequent steps.

  1. Create a self-signed root CA certificate and key.

      openssl req -new -newkey rsa:4096 -x509 -sha256 \
 -days 3650 -nodes -out relay-root-ca.crt -keyout relay-root-ca.key \
 -subj "/CN=relay-root-ca" \
 -addext "keyUsage = keyCertSign"

  2. If it doesn’t already exist, create the gloo-mesh namespace in each cluster.

      kubectl create namespace gloo-mesh --context $MGMT_CONTEXT
kubectl create namespace gloo-mesh --context $REMOTE_CONTEXT1
kubectl create namespace gloo-mesh --context $REMOTE_CONTEXT2

  3. Store the root CA certificate in the relay-root-tls-secret Kubernetes secret on the management cluster.

      kubectl create secret generic relay-root-tls-secret -n gloo-mesh \
 --from-file=tls.crt=relay-root-ca.crt \
 --from-file=ca.crt=relay-root-ca.crt \
 --from-file=tls.key=relay-root-ca.key \
 --context ${MGMT_CONTEXT}

  4. Copy the root CA certificate to each workload cluster.

      kubectl create secret generic relay-root-tls-secret -n gloo-mesh --context $REMOTE_CONTEXT1 --from-file ca.crt=relay-root-ca.crt
kubectl create secret generic relay-root-tls-secret -n gloo-mesh --context $REMOTE_CONTEXT2 --from-file ca.crt=relay-root-ca.crt

Create the intermediate CA credentials

Use the root CA key to generate an intermediate CA certificate and key. These credentials are later used to sign client TLS certificates for the Gloo agents on each workload cluster.

  1. Create the configuration for the intermediate CA.

      cat > "relay-intermediate-ca.conf" <<EOF
[req]
req_extensions = v3_req
distinguished_name = req_distinguished_name
[req_distinguished_name]
[ v3_req ]
basicConstraints = CA:TRUE
subjectKeyIdentifier = hash
authorityKeyIdentifier = keyid:always,issuer:always
keyUsage = digitalSignature, keyEncipherment, keyCertSign
extendedKeyUsage = clientAuth, serverAuth
subjectAltName = @alt_names
[alt_names]
DNS = *.gloo-mesh
EOF

  2. Generate the private key for the intermediate CA.

      openssl genrsa -out "relay-int.key" 2048

  3. Generate the certificate signing request (CSR).

      openssl req -new -key "relay-int.key" -out relay-int.csr -subj "/CN=relay-int"

  4. Sign the CSR with the root CA key.

      openssl x509 -req \
 -days 3650 \
 -CA relay-root-ca.crt -CAkey relay-root-ca.key \
 -set_serial 0 \
 -in relay-int.csr -out relay-int.crt \
 -extensions v3_req -extfile "relay-intermediate-ca.conf"

  5. Save the intermediate CA certificate and key in the relay-tls-signing-secret Kubernetes secret on the management cluster.

      kubectl create secret generic relay-tls-signing-secret -n gloo-mesh \
 --from-file=tls.crt=relay-int.crt \
 --from-file=tls.key=relay-int.key \
 --from-file=ca.crt=relay-root-ca.crt \
 --context ${MGMT_CONTEXT}

Create the server TLS certificate

Use the root CA credentials to generate a TLS certificate for the Gloo management server.

  1. Create the configuration for the server TLS certificate with the *.gloo-mesh wildcard DNS name. Because a wildcard is used, the same certificate can later be used to configure the Gloo telemetry gateway.
      cat >"relay-server.conf" <<EOF
[req]
req_extensions = v3_req
distinguished_name = req_distinguished_name
[req_distinguished_name]
[ v3_req ]
basicConstraints = CA:FALSE
keyUsage = nonRepudiation, digitalSignature, keyEncipherment
extendedKeyUsage = clientAuth, serverAuth
subjectAltName = @alt_names
[alt_names]
DNS = *.gloo-mesh
EOF
  2. Generate the private key for the Gloo management server.
      openssl genrsa -out "relay-server.key" 2048
  3. Generate the certificate signing request for the Gloo management server.
      openssl req -new -key "relay-server.key" -out "relay-server.csr" -subj "/CN=*.gloo-mesh" -config "relay-server.conf"
  4. Use the root CA credentials that you created earlier to sign the certificate signing request and create the server TLS certificate.
      openssl x509 -req \
 -days 3650 \
 -CA relay-root-ca.crt -CAkey relay-root-ca.key \
 -set_serial 0 \
 -in relay-server.csr -out relay-server.crt \
 -extensions v3_req -extfile "relay-server.conf"
  5. Store the server TLS certificate, private key, and root CA in the relay-server-tls-secret Kubernetes secret.
      kubectl create secret generic relay-server-tls-secret -n gloo-mesh \
 --from-file=tls.crt=relay-server.crt \
 --from-file=tls.key=relay-server.key \
 --from-file=ca.crt=relay-root-ca.crt \
 --context ${MGMT_CONTEXT}

Create the telemetry pipeline credentials

  1. Use the same credentials for the Gloo telemetry gateway and store them in the gloo-telemetry-gateway-tls-secret Kubernetes secret. Using the same credentials is possible, because the configuration for the Gloo management server (relay-server.conf) used a wildcard for the DNS name.
      kubectl create secret generic gloo-telemetry-gateway-tls-secret -n gloo-mesh \
 --from-file=tls.crt=relay-server.crt \
 --from-file=tls.key=relay-server.key \
 --from-file=ca.crt=relay-root-ca.crt \
 --context ${MGMT_CONTEXT}
  2. Store the root CA certificate in the telemetry-root-secret Kubernetes secret on the management and each workload cluster so that the Gloo telemetry collector agent can verify the identity of the Gloo telemetry gateway.
      kubectl create secret generic telemetry-root-secret \
 --from-file=ca.crt=relay-root-ca.crt \
 --namespace gloo-mesh \
 --context ${MGMT_CONTEXT} 
  
      kubectl create secret generic telemetry-root-secret \
 --from-file=ca.crt=relay-root-ca.crt \
 --namespace gloo-mesh \
 --context ${REMOTE_CONTEXT1}

Create relay identity tokens

To establish initial trust between the Gloo management server and agent, you must set up your own relay identity tokens.

  1. Create an environment variable with your identity token. The token can be any string value.

      export TOKEN="<identity_token>"

  2. Store the token in the relay-identity-token-secret Kubernetes secret on the management cluster.

      kubectl create secret generic relay-identity-token-secret -n gloo-mesh --context "$MGMT_CONTEXT" --from-literal=token=$TOKEN

  3. Copy the identity token to each workload cluster that you want to register.

      kubectl create secret generic relay-identity-token-secret -n gloo-mesh --context $REMOTE_CONTEXT1 --from-literal=token=$TOKEN
kubectl create secret generic relay-identity-token-secret -n gloo-mesh --context $REMOTE_CONTEXT2 --from-literal=token=$TOKEN

Install Gloo Network

  1. Follow the Install with Helm guide to set up Gloo Network.
  2. In your Helm values file for the management server, add the following values. Note that mTLS is the default mode in Gloo Network and does not require any additional configuration.
      
glooMgmtServer:
  enabled: true
  relay:
    disableCaCertGeneration: true
    signingTlsSecret:
      name: relay-tls-signing-secret
      namespace: gloo-mesh
    tlsSecret:
      name: relay-server-tls-secret
      namespace: gloo-mesh
    tokenSecret:
      key: token
      name: relay-identity-token-secret
      namespace: gloo-mesh
telemetryCollector: 
  enabled: true
  extraVolumes: 
    - name: root-ca
      secret:
        defaultMode: 420
        optional: true
        secretName: telemetry-root-secret
    - configMap:
        items:
          - key: relay
            path: relay.yaml
        name: gloo-telemetry-collector-config
      name: telemetry-configmap
    - hostPath:
        path: /var/run/cilium
        type: DirectoryOrCreate
      name: cilium-run
telemetryGateway:
  enabled: true
  extraVolumes:
  - name: tls-keys
    secret:
      secretName:  gloo-telemetry-gateway-tls-secret
      defaultMode: 420
  - name: telemetry-configmap
    configMap:
      name: gloo-telemetry-gateway-config
      items:
        - key: relay
          path: relay.yaml
telemetryGatewayCustomization:
  disableCertGeneration: true
    Helm valueDescription
    glooMgmtServer.relay.disableCaCertGenerationDisable the generation of self-signed certificates to secure the relay connection between the Gloo management server and agent.
    glooMgmtServer.relay.signingTlsSecretAdd the name and namespace of the Kubernetes secret that holds the intermediate CA credentials that you created earlier.
    glooMgmtServer.relay.tlsSecretAdd the name and namespace of the Kubernetes secret that holds the server TLS certificate for the Gloo management server that you created earlier.
    glooMgmtServer.relay.tokenSecretAdd the name, namespace, and key of the Kubernetes secret that holds the relay identity token that you created earlier.
    telemetryGateway.extraVolumesAdd the gloo-telemetry-gateway-tls-secret-custom Kubernetes secret that you created earlier to the tls-keys volume. Make sure that you also add the other volumes to your telemetry gateway configuration.
    telemetryCollector.extraVolumesAdd the telemetry-root-secret Kubernetes secret that you created earlier to the root-ca volume. Make sure that you also add the other volumes to your telemetry collector configuration.
  3. In your Helm values file for the agent, add the following values.
      
glooAgent:
  enabled: true
  relay:
    authority: gloo-mesh-mgmt-server.gloo-mesh
    clientTlsSecretRotationGracePeriodRatio: ""
    rootTlsSecret:
      name: relay-root-tls-secret
      namespace: gloo-mesh
    tokenSecret:
      key: token
      name: relay-identity-token-secret
      namespace: gloo-mesh
telemetryCollector: 
  enabled: true
  extraVolumes: 
    - name: root-ca
      secret:
        defaultMode: 420
        optional: true
        secretName: telemetry-root-secret
    - configMap:
        items:
          - key: relay
            path: relay.yaml
        name: gloo-telemetry-collector-config
      name: telemetry-configmap
    - hostPath:
        path: /var/run/cilium
        type: DirectoryOrCreate
      name: cilium-run
    Helm valueDescription
    glooAgent.relay.rootTlsSecretAdd the name and namespace of the Kubernetes secret that holds the root CA credentials that you copied from the management cluster to the workload cluster.
    glooAgent.relay.tokenSecretAdd the name, namespace, and key of the Kubernetes secret that holds the relay identity token that you copied from the management cluster to the workload cluster.