Access log

Configure how access logs are recorded for your services. You can use access log policies to configure log collection for workloads that have injected sidecars or are standalone proxies, such as gateways.

Before you begin

  1. Complete the example setup to install Gloo Mesh, Istio, and Bookinfo in your cluster.

  2. Create the Gloo Mesh resources for this policy in the management and workload clusters.

    The following files are examples only for testing purposes. Your actual setup might vary. You can use the files as a reference for creating your own tests.

    1. Download the following Gloo Mesh resources:
    2. Apply the files to your management cluster.
      kubectl apply -f kubernetes-cluster_gloo-mesh_cluster-1.yaml --context ${MGMT_CONTEXT}
      kubectl apply -f kubernetes-cluster_gloo-mesh_cluster-2.yaml --context ${MGMT_CONTEXT}
      kubectl apply -f workspace_gloo-mesh_anything.yaml --context ${MGMT_CONTEXT}
      
    1. Download the following Gloo Mesh resources:
    2. Apply the files to your workload cluster.
      kubectl apply -f workspace-settings_bookinfo_anything.yaml --context ${REMOTE_CONTEXT1}
      

Test access logs without filters

Before you apply an access log policy, check out the Envoy access logs that are recorded by default for your workload.

  1. Create a temporary container with the curl utility in the same namespace as the ratings Bookinfo service.

    kubectl run -it -n bookinfo --context $REMOTE_CONTEXT1 curl \
      --image=curlimages/curl:7.73.0 --rm  -- sh
    
  2. From the new terminal, generate access logs by sending requests to the ratings app. Note that one request contains the foo: bar request header, and the other does not. Type exit when you're done.

    curl http://ratings:9080/ratings/1 -v
    curl http://ratings:9080/ratings/1 -v -H "foo: bar"
    
  3. View the ratings access logs.

    kubectl logs -l app=ratings -c istio-proxy -n bookinfo --context $REMOTE_CONTEXT1 
    
  4. Check the logs for the ratings app. At this point, the access logs for both requests that you sent in step 1 are recorded. For example, you should see two logs for the most recent time stamp, which might look like the following:

    [2022-06-17T00:29:40.135Z] "GET /ratings/1 HTTP/1.1" 200 - via_upstream - "-" 0 48 2 1 "-" "curl/7.73.0-DEV" "8bdf59f8-0608-4d39-ac73-268519068afb" "ratings:9080" "10.24.1.17:9080" inbound|9080|| 127.0.0.6:59801 10.24.1.17:9080 10.24.0.21:48722 outbound_.9080_._.ratings.bookinfo.svc.cluster.local default
    [2022-06-17T00:30:08.114Z] "GET /ratings/1 HTTP/1.1" 200 - via_upstream - "-" 0 48 2 2 "-" "curl/7.73.0-DEV" "248926bd-ca16-42a6-866d-d2ab5a1440ee" "ratings:9080" "10.24.1.17:9080" inbound|9080|| 127.0.0.6:47333 10.24.1.17:9080 10.24.0.21:48672 outbound_.9080_._.ratings.bookinfo.svc.cluster.local default
    

    For more information about the default log format, see the Istio default access log docs. To understand each field in the default format, see the Envoy access log docs.

Configure access log policies

You can apply an access log policy at the workload level to filter the logs that you want to record.

Review the following sample configuration file.

apiVersion: observability.policy.gloo.solo.io/v2
kind: AccessLogPolicy
metadata:
  name: access-log-policy
  namespace: bookinfo
spec:
  applyToWorkloads:
  - selector:
      cluster: cluster-1
      labels:
        app: reviews
      namespace: bookinfo
  config:
    filters:
    #- statusCodeMatcher:
    #    value: 200
    #    comparator: EQ
    - headerMatcher:
        name: foo
        value: bar
        regex: false
        invertMatch: false
    #includedRequestHeaders:
    #  - x-user-agent
    #includedResponseHeaders:
    #  - x-server
    #includedResponseTrailers:
    #  - x-expires
    #includedFilterStateObjects:
Review the following table to understand this configuration.
Setting Description
spec.applyToWorkloads Configure which workloads to apply the policy to by using labels. Workloads can be apps that have injected sidecars, such as deployments or stateful sets, or standalone proxies, such as gateways. If omitted, the policy applies to all workloads in the workspace.
spec.config.filters Configure criteria for determining which access logs are recorded for the workload. The list is disjunctive, meaning that a request is logged if it matches any filter. If empty, all access logs are recorded.
spec.config.filters.statusCodeMatcher.value HTTP status codes to record logs for. The example matches for 200 response codes.
spec.config.filters.statusCodeMatcher.comparator The comparison type to match status codes. Available options: EQ for strict equality, GE for greater than or equal to, or LE for less than or equal to. The example matches logs for exactly 200 response codes.
spec.config.filters.headerMatcher HTTP headers to record logs for. For information about each field, see Header matching. The example matches logs that contain the header foo: bar.
spec.config.includedRequestHeaders For requests that match the filters, include these request headers in the recorded logs. Headers are specified by their key names. The example includes the x-user-agent request header field in the access logs.
spec.config.includedResponseHeaders For responses that match the filters, include these response headers in the recorded logs. Headers are specified by their key names. The example includes the x-server response header field in the access logs.
spec.config.includedResponseTrailers For responses that match the filters, include these response trailers in the recorded logs. Trailers are specified by their key names. The example includes the x-expires response trailer field in the access logs.
spec.config.includedFilterStateObjects For requests or responses that match the filter state object, include these filter states in the recorded logs.

For more information, see the Gloo Mesh API docs for access log policies.

Verify access log filters

After you apply the access log policy in the workload cluster, verify that logs are recorded only for requests that contain the foo: bar header.

  1. Create a temporary container with the curl utility in the same namespace as the ratings Bookinfo service.

    kubectl run -it -n bookinfo --context $REMOTE_CONTEXT1 curl \
      --image=curlimages/curl:7.73.0 --rm  -- sh
    
  2. From the new terminal, send requests to the ratings app with and without the header that you specified in the access policy. Type exit when you're done.

    curl http://ratings:9080/ratings/1 -v
    curl http://ratings:9080/ratings/1 -v -H "foo: bar"
    
  3. View the ratings access logs.

    kubectl logs -l app=ratings -c istio-proxy -n bookinfo --context $REMOTE_CONTEXT1 
    
  4. Check the logs for the ratings app to ensure that only access logs for the request that contained the foo: bar header is recorded. For example, you should only see one log for the most recent time stamp, which might look like the following:

    [2022-06-17T00:37:47.396Z] "GET /ratings/1 HTTP/1.1" 200 - via_upstream - "-" 0 48 1 1 "-" "curl/7.73.0-DEV" "f8cb715c-ffba-4b21-88dc-38e5d96d07b2" "ratings:9080" "10.24.1.17:9080" inbound|9080|| 127.0.0.6:51901 10.24.1.17:9080 10.24.0.22:32850 outbound_.9080_._.ratings.bookinfo.svc.cluster.local default
    

    For more information about the default log format, see the Istio default access log docs. To understand each field in the default format, see the Envoy access log docs.

Next steps

For more information about access logs, such as configuring the default log format, see View access logs.