Deployment model

A production Gloo Mesh Core setup consists of one management cluster that the Gloo Mesh Core management components are installed in, and one or more workload clusters that run service meshes which are registered with and managed by Gloo Mesh Core. The management cluster serves as the control plane, and the workload clusters serve as the data plane.

In a production deployment, you typically want to avoid installing the control plane into a workload cluster that also runs a service mesh. Although Gloo Mesh Core remains fully functional when the management and agent components both run within the same cluster, you might have noisy neighbor concerns in which workload pods consume cluster resources and potentially constrain the management processes. This constraint on management processes can in turn affect other workload clusters that the management components oversee. However, you can prevent resource consumption issues by using Kubernetes best practices, such as node affinity, resource requests, and resource limits.

Control plane settings

Before you install the Gloo Mesh Core control plane into your management cluster, review the following options to help secure your installation. Each section details the benefits of the security option, and the necessary settings to specify in a Helm values file to use during your Helm installation.

Certificate management

When you install Gloo Mesh Core by using meshctl or the instructions that are provided in the getting started guide, Gloo Mesh Core generates a self-signed root CA certificate and key that is used to generate the server TLS certificate for the Gloo management server. In addition, an intermediate CA certificate and key are generated that are used to sign client TLS certificates for every Gloo agent. For more information about the default setup, see Option 2: Gloo Mesh Core self-signed CAs with automatic client certificate rotation.

Using self-signed certificates and keys for the root CA and storing them on the management cluster is not a recommended security practice. The root CA certificate and key is very sensitive information, and, if compromised, can be used to issue certificates for all agents in a workload cluster. In a production-level setup you want to make sure that the root CA credentials are properly stored with your preferred PKI provider, such as AWS Private CA, Google Cloud CA, or Vault and that you use a certificate management tool, such as cert-manager to automate the issuing and renewing of certificates.

Use the following links to learn about your certificate setup options in production:

Deployment and service overrides

In some cases, you might need to modify the default deployment of the glooMgmtServer with your own Kubernetes resources. You can specify resources and annotations for the management server deployment in the glooMgmtServer.deploymentOverrides field, and resources and annotations for the service that exposes the deployment in the glooMgmtServer.serviceOverrides field.

Most commonly, the serviceOverrides section specifies cloud provider-specific annotations that might be required for your environment. For example, the following section applies the recommended Amazon Web Services (AWS) annotations for modifying the created load balancer service.

  
glooMgmtServer:
  serviceOverrides:
    metadata:
      annotations:
        # AWS-specific annotations
        service.beta.kubernetes.io/aws-load-balancer-healthcheck-healthy-threshold: "2"
        service.beta.kubernetes.io/aws-load-balancer-healthcheck-unhealthy-threshold: "2"
        service.beta.kubernetes.io/aws-load-balancer-healthcheck-interval: "10"
        service.beta.kubernetes.io/aws-load-balancer-healthcheck-port: "9900"
        service.beta.kubernetes.io/aws-load-balancer-healthcheck-protocol: "tcp"
        service.beta.kubernetes.io/aws-load-balancer-type: external
        service.beta.kubernetes.io/aws-load-balancer-scheme: internal
        service.beta.kubernetes.io/aws-load-balancer-nlb-target-type: ip
        service.beta.kubernetes.io/aws-load-balancer-backend-protocol: TCP
        service.beta.kubernetes.io/aws-load-balancer-private-ipv4-addresses: 10.0.50.50, 10.0.64.50
        service.beta.kubernetes.io/aws-load-balancer-subnets: subnet-0478784f04c486de5, subnet-09d0cf74c0117fcf3
        service.beta.kubernetes.io/aws-load-balancer-target-group-attributes: deregistration_delay.connection_termination.enabled=true,deregistration_delay.timeout_seconds=1
  # Kubernetes load balancer service type
  serviceType: LoadBalancer
  ...
  

In less common cases, you might want to provide other resources, like a config map or service account. This example shows how you might use the deploymentOverrides to specify a config map in a volume mount.

  
glooMgmtServer:
  deploymentOverrides:
    spec:
      template:
        spec:
          volumeMounts:
            - name: envoy-config
              configMap:
                name: my-custom-envoy-config
  ...
  

FIPS-compliant image

If your environment runs workloads that require federal information processing compliance, you can use images of Gloo Mesh Core components that are specially built to comply with NIST FIPS. Open the values.yaml file, search for the image section, and append -fips to the tag, such as in the following example.
  ...
glooMgmtServer:
  image:
    pullPolicy: IfNotPresent
    registry: gcr.io/gloo-mesh
    repository: gloo-mesh-mgmt-server
    tag: 2.5.0-fips
  

Licensing

During installation, you can provide your license key strings directly in license fields such as glooMeshCoreLicenseKey. For a more secure setup, you might want to provide those license keys in a secret instead.

  1. Before you install Gloo Mesh Core, create a secret with your license keys in the gloo-mesh namespace of your management cluster.
      kubectl create ns gloo-mesh
    cat << EOF | kubectl apply -n gloo-mesh -f -
    apiVersion: v1
    kind: Secret
    type: Opaque
    metadata:
      name: license-secret
      namespace: gloo-mesh
    data:
      gloo-mesh-core-license-key: ""
      gloo-trial-license-key: ""
    EOF
      
  2. When you install the Gloo Mesh Core control plane in your management cluster, specify the secret name as the value for the licensing.licenseSecretName field in your Helm values file.

Management server replicas

Horizontal replica scaling enables both resilience and distributed scaling by increasing the replica count of the management server deployment within one Kubernetes cluster. Each replica handles a subset of connected agents in workload clusters, providing scalability. If a replica pod fails, the agents that were connected to that management server replica automatically connect to a different replica. The management server deployment can span multiple availability zones (depending on the hosting topology), which can provide resilience.

By default, the management server is deployed with one replica. To increase availability, you can increase the number of replicas that you deploy in the management cluster by using the following Helm settings in your control plane deployment.

  
glooMgmtServer:
  deploymentOverrides:
    spec:
      # Required. Increase as needed.
      replicas: 2
  # Required. Mark one replica as the active or leader replica.
  leaderElection: true
  # Optional (experiemental" %}}). Attempt to evenly load balance the number of
  # connected Gloo agents across the number of management server replicas.
  enableClusterLoadBalancing: true
  

Prometheus metrics

By default, a Prometheus instance is deployed with the control plane Helm chart to collect metrics for the Gloo Mesh Core management server. For a production deployment, you can either replace the built-in Prometheus server with your own instance, or remove high cardinality labels. For more information on each option, see Customization options.

Redis instance

By default, a Redis instance is deployed for certain control plane components, such as the Gloo management server and Gloo UI. For a production deployment, you can disable the default Redis deployment and provide your own backing database instead.

For more information, see Backing databases.

UI authentication

The Gloo UI supports OpenID Connect (OIDC) authentication from common providers such as Google, Okta, and Auth0. Users that access the UI will be required to authenticate with the OIDC provider, and all requests to retrieve data from the API will be authenticated.

You can configure OIDC authentication for the UI by providing your OIDC provider details in the glooUi section, such as the following.

  ...
glooUi:
  enabled: true
  auth:
    enabled: true
    backend: oidc
    oidc:
      appUrl: # The URL that the UI for the OIDC app is available at, from the DNS and other ingress settings that expose the OIDC app UI service.
      clientId: # From the OIDC provider
      clientSecret: # From the OIDC provider. Stored in a secret.
      clientSecretName: dashboard
      issuerUrl: # The issuer URL from the OIDC provider, usually something like 'https://<domain>.<provider_url>/'.
  

Data plane settings

Before you register workload clusters with Gloo Mesh Core, review the following options to help secure your registration. Each section details the benefits of the security option, and the necessary settings to specify in a Helm values file to use during your Helm registration.

Certificate management

If you use the default self-signed certificates during Gloo Mesh Core installation, you can follow the steps in the cluster registration documentation to use these certificates during cluster registration. If you set up Gloo Mesh Core without secure communication for quick demonstrations, include the --set insecure=true flag during registration. Note that using the default self-signed certificate authorities (CAs) or using insecure mode are not suitable for production environments.

In production environments, you use the same custom certificates that you set up for Gloo Mesh Core installation during cluster registration:

  1. Ensure that when you installed Gloo Mesh Core, you set up the relay certificates, such as with AWS Certificate Manager, HashiCorp Vault, or your own custom certs, including the relay forwarding and identity secrets in the management and workload clusters.

  2. The relay certificate instructions include steps to modify your Helm values file to use the custom CAs, such as in the following relay section. Note that you might need to update the clientTlsSecret name and rootTlsSecret name values, depending on your certificate setup.

      common:
      insecure: false
    glooAgent:
      insecure: false
      relay:
        authority: gloo-mesh-mgmt-server.gloo-mesh
        clientTlsSecret:
          name: gloo-mesh-agent-$REMOTE_CLUSTER-tls-cert
          namespace: gloo-mesh
        rootTlsSecret:
          name: relay-root-tls-secret
          namespace: gloo-mesh
        serverAddress: $MGMT_SERVER_NETWORKING_ADDRESS
    ...
      

FIPS-compliant image

If your environment runs workloads that require federal information processing compliance, you can use images of Gloo Mesh Core components that are specially built to comply with NIST FIPS. Open the values.yaml file, search for the image section, and append -fips to the tag, such as in the following example.
  ...
glooAgent:
  image:
    pullPolicy: IfNotPresent
    registry: gcr.io/gloo-mesh
    repository: gloo-mesh-agent
    tag: 2.5.0-fips
  

Kubernetes RBAC

For information about controlling access to your Gloo resources with Kubernetes role-based access control (RBAC), see User access.

To review the permissions of deployed Gloo components such as the management server and agent, see Gloo component permissions.