Access log

Configure how access logs are recorded for your services. You can use access log policies to configure log collection for workloads that have injected sidecars or are standalone proxies, such as gateways.

Before you begin

This guide assumes that you use the same names for components like clusters, workspaces, and namespaces as in the getting started. If you have different names, make sure to update the sample configuration files in this guide.
  1. Complete the multicluster getting started guide to set up the following testing environment.
    • Three clusters along with environment variables for the clusters and their Kubernetes contexts.
    • The Gloo Platform CLI, meshctl, along with other CLI tools such as kubectl and istioctl.
    • The Gloo management server in the management cluster, and the Gloo agents in the workload clusters.
    • Istio installed in the workload clusters.
    • A simple Gloo workspace setup.
  2. Install Bookinfo and other sample apps.
  3. Enable access logging by modifying your default Istio operator installation.

Test access logs without filters

Before you apply an access log policy, check out the Envoy access logs that are recorded by default for your workload.

  1. Create a temporary container with the curl utility in the same namespace as the reviews Bookinfo service.

    kubectl run -it -n bookinfo --context $REMOTE_CONTEXT1 curl \
      --image=curlimages/curl:7.73.0 --rm  -- sh
    
  2. From the new terminal, generate access logs by sending requests to the reviews app. Note that one request contains the foo: bar request header, and the other does not. Type exit when you're done.

    curl http://reviews:9080/reviews/1 -v
    curl http://reviews:9080/reviews/1 -v -H "foo: bar"
    
  3. View the reviews access logs.

    kubectl logs -n bookinfo deploy/reviews-v1 -c istio-proxy --context $REMOTE_CONTEXT1
    
  4. Check the logs for the reviews app. At this point, the access logs for both requests that you sent in step 1 are recorded. For example, you should see two logs for the most recent time stamp, which might look like the following:

    {
      "requested_server_name": "outbound_.9080_._.reviews.bookinfo.svc.cluster.local",
      "path": "/reviews/1",
      "response_code": 200,
      "start_time": "2022-09-02T14:58:22.473Z",
      ...
    }
    {
      "requested_server_name": "outbound_.9080_._.reviews.bookinfo.svc.cluster.local",
      "path": "/reviews/1",
      "response_code": 200,
      "start_time": "2022-09-02T15:04:17.113Z",
      ...
    }
       

    For more information about the default log format, see the Istio default access log docs. To understand each field in the default format, see the Envoy access log docs.

Configure access log policies

You can apply an access log policy at the workload level to filter the logs that you want to record.

Review the following sample configuration file.

apiVersion: observability.policy.gloo.solo.io/v2
kind: AccessLogPolicy
metadata:
  name: access-log-policy
  namespace: bookinfo
spec:
  applyToWorkloads:
  - selector:
      cluster: cluster1
      labels:
        app: reviews
      namespace: bookinfo
  config:
    filters:
    #- statusCodeMatcher:
    #    value: 200
    #    comparator: EQ
    - headerMatcher:
        name: foo
        value: bar
        regex: false
        invertMatch: false
    #includedRequestHeaders:
    #  - x-user-agent
    #includedResponseHeaders:
    #  - x-server
    #includedResponseTrailers:
    #  - x-expires
    #includedFilterStateObjects:
Review the following table to understand this configuration.
Setting Description
spec.applyToWorkloads Configure which workloads to apply the policy to by using labels. Workloads can be apps that have injected sidecars, such as deployments or stateful sets, or standalone proxies, such as gateways. If omitted, the policy applies to all workloads in the workspace.
spec.config.filters Configure criteria for determining which access logs are recorded for the workload. The list is disjunctive, meaning that a request is logged if it matches any filter. If empty, all access logs are recorded.
spec.config.filters.statusCodeMatcher.value HTTP status codes to record logs for. The example matches for 200 response codes.
spec.config.filters.statusCodeMatcher.comparator The comparison type to match status codes. Available options: EQ for strict equality, GE for greater than or equal to, or LE for less than or equal to. The example matches logs for exactly 200 response codes.
spec.config.filters.headerMatcher HTTP headers to record logs for. For information about each field, see Header matching. The example matches logs that contain the header foo: bar.
spec.config.includedRequestHeaders For requests that match the filters, include these request headers in the recorded logs. Headers are specified by their key names. The example includes the x-user-agent request header field in the access logs.
spec.config.includedResponseHeaders For responses that match the filters, include these response headers in the recorded logs. Headers are specified by their key names. The example includes the x-server response header field in the access logs.
spec.config.includedResponseTrailers For responses that match the filters, include these response trailers in the recorded logs. Trailers are specified by their key names. The example includes the x-expires response trailer field in the access logs.
spec.config.includedFilterStateObjects For requests or responses that match the filter state object, include these filter states in the recorded logs.

For more information, see the Gloo Mesh API docs for access log policies.

Verify access log filters

After you apply the access log policy in the workload cluster, verify that logs are recorded only for requests that contain the foo: bar header.

  1. Create the access log policy, such as the configuration file that you previously reviewed.

    kubectl apply -f access-log-policy.yaml --context $REMOTE_CONTEXT1
    
  2. Create a temporary container with the curl utility in the same namespace as the reviews Bookinfo service.

    kubectl run -it -n bookinfo --context $REMOTE_CONTEXT1 curl \
      --image=curlimages/curl:7.73.0 --rm  -- sh
    
  3. From the new terminal, send requests to the reviews app with and without the header that you specified in the access policy. Type exit when you're done.

    curl http://reviews:9080/reviews/1 -v
    curl http://reviews:9080/reviews/1 -v -H "foo: bar"
    
  4. View the reviews access logs.

    kubectl logs -n bookinfo deploy/reviews-v1 -c istio-proxy --context $REMOTE_CONTEXT1 
    
  5. Check the logs for the reviews app to ensure that only access logs for the request that contained the foo: bar header is recorded. For example, you should only see one log for the most recent time stamp, which might look like the following:

     
    {
      "requested_server_name": "outbound_.9080_._.reviews.bookinfo.svc.cluster.local",
      "path": "/reviews/1",
      "response_code": 200,
      "start_time": "2022-09-02T14:58:22.473Z",
      ...
    }
       

    For more information about the default log format, see the Istio default access log docs. To understand each field in the default format, see the Envoy access log docs.

  6. Optional: Clean up the resources that you created.

    kubectl --context $REMOTE_CONTEXT1 -n bookinfo delete AccessLogPolicy access-log-policy
    

Next steps

For more information about access logs, such as configuring the default log format, see View access logs.