Relay architecture

The Gloo architecture consists of a Gloo management server that is commonly installed in a management cluster, and Gloo agents that are deployed to workload clusters. The workload cluster is registered with the management cluster and hosts your apps and services. All Gloo Platform products share this architecture to provide exceptional routing, discovery, security, scalability, and observability capabilities for your apps across clusters, clouds, and environments.

In testing or single-cluster environments, you can also run the Gloo control plane and data plane components in the same cluster.

Gloo components

Gloo consists of a management server (sometimes called the “relay server”) for the control plane and relay agents for each workload cluster in the data plane.

Management server

After installation, a deployment named gloo-mesh-mgmt-server with multiple replicas runs the management server. The management server is exposed by the gloo-mesh-mgmt-server service on a default port of 9900/TCP. The server is responsible for configuring the Gloo agents in your workload cluster and maintaining the desired state of your environment. When you create Gloo custom resources, the server translates these to the appropriate open source custom resources that your Gloo product is based on, such as Istio, Envoy, or Cilium. Then, the server pushes config changes to the agents to apply in the workload clusters.

For your workload clusters to communicate with the management server, the gloo-mesh-mgmt-server service is automatically set up as a service of type LoadBalancer.

Relay agents

After registration, a deployment named gloo-mesh-agent runs the relay agent on each workload cluster. The relay agent is exposed by the gloo-mesh-agent service on the default port 9977. The agents send product-specific metrics and snapshots of the resources from each workload cluster to the management server.

Agent replicas

You can add replicas of the agent for higher availability. In such case, leader election affects which processes the agents handle. Logging and metric processes use the most resources, and scale as the number of services within the cluster grows.

The leader agent handles the following processes:

All agent replicas, including the leader, handle the following processes:

Agent-server communication

Communication between the management server and agents is initiated by the Gloo relay agents, which run in the workload clusters. The following figures outline the general flow of how the relay agents and server communicate to keep your configurations and ennvironment up to date.

Note that these steps outline the relay process in a multicluster setup.

Management server and relay agent registration

A workload cluster is registered with the Gloo management server in the control plane. The relay agent in the workload cluster establishes two-way, mTLS-secured gRPC communication with the management server in the management cluster.

Note that for high availability, multiple instances of the gloo-mesh-mgmt-server pod run the management server. Each relay agent connects and sends its data to only one of the instances. To translate the resources that a server instance receives from the agents that are connected to it, the server instance receives:

Figure: Relay agents on workload clusters are registered with the Gloo management server on the management cluster.

Agent snapshot and product-specific metrics

The relay agent in the workload cluster sends a snapshot of its state to the management server. The following types of resources are included in the snapshot:

In addition, product-specific metrics are collected from the apps in your workload clusters and sent to the Gloo agent via gRPC push procedures. The agent then forwards the metrics to the Gloo management server where they are scraped by the built-in Prometheus server. To access the metrics, you can use PromQL queries in the Prometheus UI, or view the metrics graph in the Gloo UI.

Figure: Relay agents discover resources in the workload cluster, and send a snapshot of resource and cluster states to the management server.

Gloo custom resource translation

When you create a configuration, the management server translates the configuration into other Gloo custom resources or resources that are specific to the open source project that the Gloo product is based on, such as Istio, Envoy, or Cilium. These resources form a complete view of your Gloo environment and are stored in the built-in Redis database.

The Gloo agent pulls translated custom resources as a snapshot from the Gloo management server, and applies the resources in the workload cluster. In a single cluster setup, these resources are written directly to the cluster without relay. For more information about how Gloo translates custom resources, see Custom resource translation.

Figure: The management server translates the resource snapshots into Istio configurations, and relay agents pull the Istio configurations into each cluster.

More details about agent-server communication

Now that you reviewed the overall flow of how Gloo's relay architecture works, learn more about how Gloo secures communication, translates Gloo resources, and reconciles configuration updates.

Secure communication between relay agents and management server

Communication between the relay server and agent uses the gRPC protocol, and is secured by mTLS. When you install Gloo in the management server, a relay-identity-token-secret is created for you. You copy this secret as part of registering a workload cluster. The relay-identity-token-secret on each workload cluster must match the management cluster. To validate authenticity during registration, the agent uses simple TLS to send the token to the management server. After validating the token, the management server creates a TLS certificate for the relay agent. Then, all future communication from relay agents to the server uses this certificate for mTLS.

Note that you can use self-signed certificates or certificates from your PKI to secure server-agent communication. For more information, see the certificate management strategies for proof-of-concept or production.

Resource discovery by relay agents

Each relay agent performs resource discovery for the cluster that it is deployed to. The relay agent constructs a snapshot of the actual state of discovered entities in the workload cluster, including the following.

The resources in this snapshot provide the management server with the complete state of the cluster, workloads, and destinations.

Configuration updates and state reconciliation

The relay agents watch for user-provided configuration updates in your workload clusters. For example, you might create a Gloo policy in one of your workload clusters. Relay agents then create a snapshot of all Gloo and open source custom resources in the workload cluster, and send the snapshot to the management server in the management cluster. The management server uses this information to decide what custom resources must be created. For example, if you use Gloo Mesh or Gloo Gateway, your Gloo policy might be translated into a VirtualService, DestinationRule, and AuthorizationPolicy Istio resources.

The management server then reconciles this declared state with the actual state of the workload clusters, and creates configuration updates. The relay agents pull these updates in real time from the management server. The agents apply the updated resources in each cluster.