The Gloo Mesh API helps you set up and manage an Istio service mesh in a single or many Kubernetes clusters.
Learn about the core concepts behind Gloo Mesh, including Kubernetes, Envoy, and Istio.
Before diving into how the Gloo Mesh API is organized and designed, flip through the following core concepts.
- Apps as microservices
- Kubernetes cluster
- Envoy proxy
- Istio service mesh
- Gloo Mesh custom resources
- Gloo Mesh workspaces
You might have heard a lot of talk about the “modernization” of monolithic apps to microservices. For example, your team might write all of your applications as a single unit, or “monolithic” app. Deploying the app is simple, as long as you knew the app requirements and your environment. Securing the app is also simple, because you typically control a single ingress and egress point.
As your team adds more features, however, your monolithic app can become slower and harder to update and scale. Additionally, you duplicate a lot of business logic for management features like connectivity, security, reliability, and observability, into each monolithic app.
You can develop faster if you split up the larger app into smaller apps whose services can be consumed by other services on-demand. This microservice approach gives your teams more autonomy and flexibility, but adds a level of management complexity. Open source projects, such as Kubernetes and Istio, help solve the microservice management problem.
Kubernetes is a platform that helps you manage containerized applications at scale. You start with Nodes, which might be bare metal, virtual machines (VMs), or other types of computing hosts. Then, you group the nodes together into a Kubernetes cluster. Finally, you use Kubernetes API objects to run your containerized apps in the cluster.
Usually, you configure Kubernetes API objects with YAML files that conform to various schemas. For example, you can package a container in several different types of workload objects, the smallest of which is a Pod. Another common object is a Kubernetes Service, which exposes the container of a particular workload object. A service makes the app available to other apps within the cluster, or more publicly outside the cluster on a specific port or load balancer.
Resources are organized into namespaces for isolation. Often, users can configure resources only within the namespaces that they have access to. You might also set rules for compute resource consumption by namespace.
Although Kubernetes ships with some very powerful primitives, you can also extend the platform with custom resource definitions (CRDs). After installing CRDs, you can use Kubernetes-native API and CLI tooling to manage those custom resources. For example, Gloo Mesh installs CRDs so that you can create and manage Gloo Mesh resources such as virtual meshes and workspaces.
Envoy is an open source proxy. The proxy, or API gateway, is a server that handles requests to and responses from your apps. You can put a lot of the business logic that you used to have in a monolithic app into the proxy. Then, all your microservices can share this logic by using the same proxy.
The Envoy proxy supports Layer 4 transport and Layer 7 application data of the OSI model. Envoy has many capabilities to make your application traffic more performant, extensible, observable, and secure. For example, Envoy can get real-time configuration updates through the xDS API. These updates do not require a proxy restart. You can adjust available routes and policies without downtime.
Istio is an open source service mesh that you install in Kubernetes clusters. A service mesh is a set of networking tools to help you manage traffic across all of the services in your clusters.
First, Istio updates the pods in your cluster to have an additional container, or “sidecar,” which is an Envoy proxy. The sidecar intercepts the requests to and responses from the pod. The sidecars are controlled by the
istiod deployment per cluster. You configure the sidecar proxy through various custom Istio resources. For example, a VirtualService resource controls routing and can add some networking policies such as automatic retries and timeout limits.
A typical order of operations looks like the following.
Productsmakes a request to
Productssidecar intercepts the request.
Productssidecar sends the request to the
Reviewssidecar sends the request to
Reviewssends a response that is handled by its sidecar.
Reviewssidecar sends the response to the
Productssidecar sends the response to
This traffic across services in the service is also known as east-west traffic.
Istio also provides an ingress gateway, which is a standalone instance of Envoy that manages traffic in and out of the cluster. Such ingress traffic is also known as north-south traffic.
As you can tell, application networking can become quite complicated! You have to know a lot about Kubernetes, Envoy, and Istio. If you want more than one cluster, such as for high availability, keeping everything in sync becomes even harder. Gloo Mesh simplifies managing your app traffic across clusters by introducing its own custom resources (CRs).
You use Gloo Mesh custom resources to group together all of your service meshes into a virtual mesh CR. Then, you create other Gloo Mesh networking and policy CRs to control traffic consistently across service meshes in different clusters.
For more information, see the following resources.
- Learn more about Gloo Mesh benefits in the Benefits page.
- Configure each custom resource in the API reference documentation.
Gloo introduces a new concept for Kubernetes-based multitenancy, the Workspace custom resource. A workspace consists of one or more Kubernetes namespaces that are in one or more clusters. Think of a workspace as the boundary of your team's resources. To get started, you can create a workspace for each of your teams. Your teams might start with their apps in a couple Kubernetes namespaces in a single cluster. As your teams scale across namespaces and clusters, their workspaces scale with them.
Gloo Mesh can help you by organizing your namespaces into Workspaces. You can share resources across workspaces by configuring the workspace settings to import and export. To prevent unintentional sharing, both workspace settings must have a matching import-export setup. You can even select particular resources to import or export, such as with Kubernetes labels, names, or namespaces. Your developers do not have to worry about exposing their apps. They deploy the app in their own namespace, and refer to the services they need to use, even if the service is in another namespace or workspace. As long as the appropriate Gloo Mesh CRs are in an imported workspace, Gloo Mesh takes care of the rest.
These CRs might be networking or policy resources that manage traffic within the workspace. These capabilities make it easier for you to onboard teams and increase consistency in your multitenant environment. With workspaces, you also can more easily control access to your services through consistent security policies.
For more information, see the following resources.
The v2 API is organized into groups. Review the following table and diagram for a brief overview of the groups of APIs.
You can learn more about specific APIs in the API reference documentation.
|API group||API endpoint||Description|
||The v2 API introduces a new set of admin APIs, including Workspaces. Use these APIs to configure how your team uses the service mesh across clusters. For example, use workspaces as team boundaries across Kubernetes clusters and namespaces. Decide which workspaces can import and reuse resources from other workspaces. Set up the ability to deploy extra servers for rate limiting or external auth. Then, you can delegate administration so that each team only accesses what they need to.|
||Use the infrastructure APIs to configure interaction between resources in your Gloo Platform setup and resources in your cloud infrastructure provider. For example, you can create a cloud provider custom resource to define the AWS region, account, and IAM roles that you want Gloo Gateway to use to access AWS Lambda functions.|
||Use the networking APIs to set up connectivity across services. For example, you can create virtual gateways that use route tables with the host domains of your services. You can even route to services outside a Kubernetes cluster with external destinations. For example, you might have an app in a bare metal or virtual machine. Your networking resources reside in your workspaces for more control over your services.|
||Apply policies to routes or destinations within your workspace for more granular control. The policies help you control resilience, security, observability, traffic, and more. Any service in the workspace can reuse the policies you want. You can even export policies to other workspaces, making policies consistent across teams.|
Using Gloo Mesh API v2
- Install Gloo Mesh Enterprise in each management and workload cluster to at least version 2.0.
- Review the API groups of objects that you can create with API v2.
- When you create the custom resources, make sure to specify the API v2 endpoint in the configuration files. For example, you can create a global workspace with the following configuration.
apiVersion: admin.gloo.solo.io/v2 kind: Workspace metadata: annotations: cluster.solo.io/cluster: "" name: anything namespace: gloo-mesh spec: workloadClusters: - name: '*' namespaces: - name: '*'
- Develop a labelling strategy for your resources. The Gloo Mesh API relies on selectors to choose objects. For example, labels select the policies to apply to resources like virtual destinations. For more information about labels and selectors, see the Kubernetes documentation.
Creating Gloo custom resources
Review the following guidance for creating Gloo custom resources.
What Gloo resources can I create?
After installing Gloo in your cluster, you can list available custom resources with the following command.
kubectl api-resources -o wide | grep solo
To learn more about each resource, review the API reference docs.
How do I create Gloo resources?
Create a configuration file, typically in YAML format. To help build your files, the docs provide many copiable examples. Then, review the API reference docs for more in-depth information on specific settings.
Where should I create Gloo resources?
With Gloo, you can set up multitenant workspaces that reflect how you organize your team. Very few custom resources are required to be created in particular cluster or namespace. However, you might prefer to create some custom resources together for simplicity.
Simple setup: Create all your Gloo resources in the management cluster. This way, your workload clusters are reserved for your apps, and all of the networking configuration is stored on the same management cluster. You can still set up multiple workspaces to keep each team's configuration separate from each other.
Gloo provides the flexibility for you to create custom resources in workload clusters, too. This setup adds more complexity in terms of where resources live. On the other hand, you can offload the platform administrator's work by giving each team the ability to create and manage their own resources in their own clusters.
Custom resource location requirements:
KubernetesClusterresources must be created in the management cluster.
Workspaceresources must be created in the management cluster.
WorkspaceSettingsmust be created in a namespace and cluster that is part of the workspace. A workspace can only have one
Cluster and namespace considerations:
- The management server and agent are expected to be created in the
gloo-meshnamespace. If you want to create them in a different namespace, review their configuration for instances of
gloo-mesh. For example, each agent's relay certificate authority field
authority: gloo-mesh-mgmt-server.gloo-meshmust be updated to the new namespace,
- Resources that refer to another resource often default to assuming that the other resource is in the same namespace and cluster, such as an external auth policy that selects an external auth server or an OAuth secret. If the resources are not in the same location, make sure to update the configuration.
How can I share Gloo resources across workspaces and clusters?
To share resources, you set up importing and exporting at the workspace level. Review the following table to learn how where you create a resource restricts how you can share the resource. For more information, see Import and export resources across workspaces.
|Where you want the resource available||Where to create the resource|
|Only to resources in the same workspace||Any namespace in the workspace. Include a virtual destination for multicluster workspaces. Do not export the workspace to other workspaces.|
|Only to resources in the same workspace on the same cluster||Any namespace in the cluster in the workspace. Do not include a virtual destination. Do not export the workspace to other workspaces.|
|To resources in other workspaces||Anywhere in your workspace. Make sure that your workspace exports to the other workspaces. The other workspaces also must import your workspace. Consider adding labels to share workspace resources more scalably.|
How can I apply policies to my app network traffic?
Policies might apply to the following resources:
- Destinations: Destinations determine where incoming requests are routed to. A destination can be a Kubernetes Service, a Gloo VirtualDestination in a multicluster setup, or a Gloo ExternalService custom resource that you use to reach endpoints that are outside your service mesh.
- Listeners: Listeners are the virtual gateways and ports that a gateway workload listens for incoming traffic on. You must apply a listener policy to a Gloo VirtualGateway custom resource, not the gateway workload directly.
- Route: Routes define how to match an incoming request, and what actions you want to perform on a matched request. For example, you can decide to forward the request to a destination in the service mesh, directly respond to the request without forwarding, or manipulate and redirect requests. Routes are defined in a Gloo RouteTable custom resource.
- Workloads: A workload represents the app that responds to incoming request. For example, you can have Kubernetes workloads such as Deployments or StatefulSets. Or, you can use an Istio WorkloadEntry that might run in a bare metal or virtual machine outside your Kubernetes cluster.
You can apply the policies by using Kubernetes labels and selectors that match the route table, virtual destination, or workload. Remember, all of these resources must be in the same workspace for the policy to apply the resource. To see what resources a policy might select, check the Kubernetes labels such as with the following commands.
kubectl get <KIND> <RESOURCE> -n <NAMESPACE> -l <KEY=VALUE> kubectl get all -A -l env=prod
Custom resource translation
After installing and registering clusters with Gloo, certain Kubernetes and Istio resources are automatically discovered and translated as custom resources in Gloo. Going forward, you can configure Gloo custom resources to automatically create the Istio resources that are needed to manage your Gloo environment.
Translated resources follow a naming convention of
<first-31-chars>-<32-char-hash-of-full-string> and are 63 characters or less. This convention is used because Kubernetes labels can only be up to 63 characters long, and translated names are sometimes used in labels.
The following figure outlines how Gloo Mesh custom resources are translated into Istio resources.
Note that many translations depend on fields that you configure in a Gloo Mesh resource. For example, a policy might create only a DestinationRule, or both a DestinationRule and VirtualService, depending on how you configure the policy. Additionally, multiple Gloo Mesh resources can configure the same or different Istio resources, depending on what you specify.
With a firm grounding in the core API concepts, you're ready to learn more about Gloo Mesh's architecture or dive into the Get Started guide.