Configure cross-account Lambda access

Configure Solo Enterprise for kgateway to route to AWS Lambda functions in different accounts than the AWS account that Solo Enterprise for kgateway authenticates with.

About

In some cases, you might want Solo Enterprise for kgateway to route requests to Lambda functions that exist in other AWS accounts that the account the you used to authenticate with Solo Enterprise for kgateway. For example, your organization might establish multitenancy in which each team has its own AWS account where its Lambda functions exist, but your organization uses one dedicated AWS account for authentication purposes.

You can set up multi-account routing in the following ways:

IAM Roles for Service Accounts role-chained configuration (recommended): Use AWS IAM Roles for Service Accounts (IRSA) to configure routing to functions in different accounts. This method is recommended for routing to Lambda functions across multiple accounts.
Resource-based configuration: Use AWS resource-based configuration to enable routing to functions in different accounts. A resource-based policy can give other accounts the permission to invoke a function without requiring the other accounts to assume a role.

IRSA role-chained configuration

Use AWS IAM Roles for Service Accounts (IRSA) to configure routing to functions in different accounts. This method is recommended for using cross-account Lambda functions with Solo Enterprise for kgateway.

This method requires you to enable IAM settings in your EKS cluster, such as the AWS Pod Identity Webhook, before you deploy Solo Enterprise for kgateway components that are created during the installation, such as the Gateway CRD and the gateway proxy service account. You might use these steps with a fresh EKS test cluster to try out IRSA role-chained configuration with Solo Enterprise for kgateway.

Save AWS account details

Save the ID of the AWS account that you want to use for authentication (the “authentication account”).
```
export AUTH_ACCOUNT_ID=<auth_account_id>
```
Save the region and ID of the AWS account where the Lambda functions that you want to invoke exist (the “Lambda account”).
```
export AWS_LAMBDA_REGION=<lambda_account_region>
export LAMBDA_ACCOUNT_ID=<lambda_account_id>
```
Save the region and name of the EKS cluster that you plan to install Solo Enterprise for kgateway in.
```
export AWS_CLUSTER_REGION=<cluster_region>
export CLUSTER_NAME=<cluster_name>
```

Save and verify your cluster’s OIDC provider details.

Get the OIDC provider for your cluster, in the format oidc.eks.<region>.amazonaws.com/id/<cluster_id>.
```
export OIDC_PROVIDER=$(aws eks describe-cluster --name ${CLUSTER_NAME} --region ${AWS_CLUSTER_REGION} --query "cluster.identity.oidc.issuer" --output text | sed -e "s/^https:\/\///")
echo $OIDC_PROVIDER
```
- If an OIDC provider is not returned, follow the AWS documentation to Create an IAM OIDC provider for your cluster, and then run this command again to save the OIDC provider in an environment variable.

Verify that the OIDC provider’s ARN is listed as an entry in AWS IAM.

aws iam list-open-id-connect-providers \
  --query "OpenIDConnectProviderList[?contains(Arn, '${OIDC_PROVIDER}')].Arn" \
  --output text

If your OIDC provider is not returned, add it to IAM.

aws iam create-open-id-connect-provider \
  --url "https://${OIDC_PROVIDER}" \
  --client-id-list sts.amazonaws.com \
  --thumbprint-list "$(openssl s_client -servername $(echo ${OIDC_PROVIDER} | cut -d/ -f1) -showcerts -connect $(echo ${OIDC_PROVIDER} | cut -d/ -f1):443 </dev/null 2>/dev/null | openssl x509 -fingerprint -noout -sha1 | cut -d= -f2 | tr -d ':')"

Configure AWS IAM resources

Create roles in your authentication and Lambda AWS accounts.

In the account that you want to use to authenticate with AWS (the “authentication account”), you create a role that is used to assume the role in the Lambda account.
In the account that contains the Lambda functions you want to route to (the “Lambda account”), you create a role that is used to invoke the Lambda functions.

In your authentication account, create the following resources.

Create an IAM policy to allow access to the following four Lambda actions.

cat >policy-1.json <<EOF
{
   "Version": "2012-10-17",
   "Statement": [
       {
           "Effect": "Allow",
           "Action": [
               "lambda:ListFunctions",
               "lambda:InvokeFunction",
               "lambda:GetFunction",
               "lambda:InvokeAsync"
           ],
           "Resource": "*"
       }
   ]
}
EOF

aws iam create-policy --policy-name lambda-auth-policy --policy-document file://policy-1.json

Create the following IAM role, which associates the policy with the Kubernetes service account of the HTTP gateway proxy component. Note that the service account name http in the kgateway-system namespace is specified, because in later steps you create an HTTP gateway named http.

cat >role-1.json <<EOF
{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Principal": {
        "Service": "ec2.amazonaws.com"
      },
      "Action": "sts:AssumeRole"
    },
    {
      "Effect": "Allow",
      "Principal": {
        "Federated": "arn:aws:iam::${AUTH_ACCOUNT_ID}:oidc-provider/${OIDC_PROVIDER}"
      },
      "Action": "sts:AssumeRoleWithWebIdentity",
      "Condition": {
        "StringEquals": {
          "${OIDC_PROVIDER}:sub": [
            "system:serviceaccount:kgateway-system:http"
          ]
        }
      }
    }
  ]
}
EOF

aws iam create-role --role-name lambda-auth-role --assume-role-policy-document file://role-1.json

Attach the IAM role to the IAM policy. This IAM role for the service account is known as an IRSA.

aws iam attach-role-policy --role-name lambda-auth-role --policy-arn=arn:aws:iam::${AUTH_ACCOUNT_ID}:policy/lambda-auth-policy

Verify that the policy is attached to the role.

aws iam list-attached-role-policies --role-name lambda-auth-role

Example output:

{
    "AttachedPolicies": [
        {
            "PolicyName": "lambda-auth-policy",
            "PolicyArn": "arn:aws:iam::111122223333:policy/lambda-auth-policy"
        }
    ]
}

In your Lambda account, create the following resources.

Create an IAM policy that contains at least the lambda:InvokeFunction permission to allow Lambda function invocation, such as the following example.

cat >policy-2.json <<EOF
{
   "Version": "2012-10-17",
   "Statement": [
       {
           "Effect": "Allow",
           "Action": [
               "lambda:ListFunctions",
               "lambda:InvokeFunction",
               "lambda:GetFunction",
               "lambda:InvokeAsync"
           ],
           "Resource": "*"
       }
   ]
}
EOF

aws iam create-policy --policy-name lambda-invoke-policy --policy-document file://policy-2.json

Create an IAM role that specifies the ARN of the authentication account’s role that you created in step 1.2, such as the following example. This ensures that the authentication account’s role that you previously created can assume this Lambda account role to invoke functions.

cat >role-2.json <<EOF
{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Principal": {
                "AWS": "arn:aws:iam::${AUTH_ACCOUNT_ID}:role/lambda-auth-role"
            },
            "Action": "sts:AssumeRole",
            "Condition": {}
        }
    ]
}
EOF

aws iam create-role --role-name lambda-invoke-role --assume-role-policy-document file://role-2.json

Attach the invocation IAM role to the invocation IAM policy.

aws iam attach-role-policy --role-name lambda-invoke-role --policy-arn=arn:aws:iam::${LAMBDA_ACCOUNT_ID}:policy/lambda-invoke-policy

Verify that the policy is attached to the role.

aws iam list-attached-role-policies --role-name lambda-invoke-role

Example output:

{
    "AttachedPolicies": [
        {
            "PolicyName": "lambda-invoke-policy",
            "PolicyArn": "arn:aws:iam::111122223333:policy/lambda-invoke-policy"
        }
    ]
}

Choose an existing or create a new Lambda function that you want to route to. To create a simple echo Node.js function for testing, see Create a Lambda function for testing.

Deploy the Amazon EKS Pod Identity Webhook

Before you install Gloo Gateway, deploy the Amazon EKS Pod Identity Webhook, which allows pods’ service accounts to use AWS IAM roles. When you create the Gloo Gateway proxy in the next section, this webhook mutates the proxy’s service account so that it can assume your IAM role to invoke Lambda functions.

In your EKS cluster, install cert-manager, which is a prerequisite for the webhook.

wget https://github.com/cert-manager/cert-manager/releases/download/v1.12.4/cert-manager.yaml
kubectl apply -f cert-manager.yaml

Verify that all cert-manager pods are running.
```
kubectl get pods -n cert-manager
```

Deploy the Amazon EKS Pod Identity Webhook.

kubectl apply -f https://raw.githubusercontent.com/solo-io/workshops/refs/heads/master/gloo-gateway/1-18/enterprise/lambda/data/steps/deploy-amazon-pod-identity-webhook/auth.yaml
kubectl apply -f https://raw.githubusercontent.com/solo-io/workshops/refs/heads/master/gloo-gateway/1-18/enterprise/lambda/data/steps/deploy-amazon-pod-identity-webhook/deployment-base.yaml
kubectl apply -f https://raw.githubusercontent.com/solo-io/workshops/refs/heads/master/gloo-gateway/1-18/enterprise/lambda/data/steps/deploy-amazon-pod-identity-webhook/mutatingwebhook.yaml
kubectl apply -f https://raw.githubusercontent.com/solo-io/workshops/refs/heads/master/gloo-gateway/1-18/enterprise/lambda/data/steps/deploy-amazon-pod-identity-webhook/service.yaml

Verify that the webhook deployment completes.

kubectl rollout status deploy/pod-identity-webhook

Install Solo Enterprise for kgateway

Be sure that you deployed the Amazon EKS Pod Identity Webhook to your cluster first before you continue to install Solo Enterprise for kgateway.

Set your Solo Enterprise for kgateway license key as an environment variable. If you do not have one, contact an account representative.

Deploy the Kubernetes Gateway API CRDs.

kubectl apply -f https://github.com/kubernetes-sigs/gateway-api/releases/download/v1.4.0/standard-install.yaml

Deploy the Solo Enterprise for kgateway CRDs by using Helm. The following command uses the latest stable release, 2.1.5. For active development, update the version to 2.2.0-main.

helm upgrade -i enterprise-kgateway-crds oci://us-docker.pkg.dev/solo-public/enterprise-kgateway/charts/enterprise-kgateway-crds \
  --create-namespace \
  --namespace kgateway-system \
  --version 2.1.5

Install Solo Enterprise for kgateway by using Helm. Choose between the Envoy-based kgateway proxy or the AI-first agentgateway proxy. You can also enable both gateway proxy types. Note that you need a separate license for each gateway proxy type that you want to enable.

helm upgrade -i enterprise-kgateway oci://us-docker.pkg.dev/solo-public/enterprise-kgateway/charts/enterprise-kgateway \
  -n kgateway-system \
  --version 2.1.5 \
  --set-string licensing.glooGatewayLicenseKey=$GLOO_GATEWAY_LICENSE_KEY

helm upgrade -i enterprise-kgateway oci://us-docker.pkg.dev/solo-public/enterprise-kgateway/charts/enterprise-kgateway \
  -n kgateway-system \
  --version 2.1.5 \
  --set agentgateway.enabled=true \
  --set-string licensing.agentgatewayLicenseKey=$AGENTGATEWAY_LICENSE_KEY

helm upgrade -i enterprise-kgateway oci://us-docker.pkg.dev/solo-public/enterprise-kgateway/charts/enterprise-kgateway \
  -n kgateway-system \
  --version 2.1.5 \
  --set agentgateway.enabled=true \
  --set-string licensing.glooGatewayLicenseKey=$GLOO_GATEWAY_LICENSE_KEY \
  --set-string licensing.agentgatewayLicenseKey=$AGENTGATEWAY_LICENSE_KEY

Make sure that the enterprise-kgateway control plane is running.

kubectl get pods -n kgateway-system

Example output:

NAME                            READY   STATUS    RESTARTS   AGE
enterprise-kgateway-5495d98459-46dpk   1/1     Running   0          19s

Annotate the gateway proxy service account

Create a EnterpriseKgatewayParameters resource to specify the invocation role ARN in the eks.amazonaws.com/role-arn IRSA annotation for the gateway proxy service account.

kubectl apply -f- <<EOF
apiVersion: enterprisekgateway.solo.io/v1alpha1
kind: EnterpriseKgatewayParameters
metadata:
  name: http-lambda
  namespace: kgateway-system
spec:
  kube:
    serviceAccount:
      extraAnnotations:
        eks.amazonaws.com/role-arn: arn:aws:iam::${LAMBDA_ACCOUNT_ID}:role/lambda-invoke-role
EOF

Update the http Gateway resource to add a reference to the http-lambda EnterpriseKgatewayParameters.

kubectl apply -f- <<EOF
kind: Gateway
apiVersion: gateway.networking.k8s.io/v1
metadata:
  name: http
  namespace: kgateway-system
  annotations:
spec:
  gatewayClassName: enterprise-kgateway
  infrastructure:
    parametersRef:
      name: http-lambda
      group: enterprisekgateway.solo.io
      kind: EnterpriseKgatewayParameters        
  listeners:
  - protocol: HTTP
    port: 8080
    name: http
    allowedRoutes:
      namespaces:
        from: All
EOF

Check the status of the gateway to make sure that your configuration is accepted. Note that in the output, a NoConflicts status of False indicates that the gateway is accepted and does not conflict with other gateway configuration.
```
kubectl get gateway http -n kgateway-system -o yaml
```
Verify that the http service account has the eks.amazonaws.com/role-arn: arn:aws:iam::${LAMBDA_ACCOUNT_ID}:role/lambda-invoke-role annotation.
```
kubectl describe serviceaccount http -n kgateway-system
```

Set up routing to your function

Create Backend and HTTPRoute resources to route requests to the Lambda function.

Create a Backend resource that references the AWS region, the ID of the account that contains the IAM role for Lambda invocation, and the echo function that you created.

kubectl apply -f - <<EOF
apiVersion: enterprisekgateway.solo.io/v1alpha1
kind: Backend
metadata:
  name: lambda
  namespace: kgateway-system
spec:
  type: AWS
  aws:
    region: ${AWS_LAMBDA_REGION}
    accountId: "${LAMBDA_ACCOUNT_ID}"
    lambda:
      functionName: echo
EOF

Create an HTTPRoute resource that references the lambda Backend.

kubectl apply -f - <<EOF
apiVersion: gateway.networking.k8s.io/v1
kind: HTTPRoute
metadata:
  name: lambda
  namespace: kgateway-system
spec:
  parentRefs:
    - name: http
      namespace: kgateway-system
  rules:
  - matches:
    - path:
        type: PathPrefix
        value: /echo
    backendRefs:
    - name: lambda
      namespace: kgateway-system
      group: enterprisekgateway.solo.io
      kind: Backend
EOF

Get the external address of the gateway and save it in an environment variable.

export INGRESS_GW_ADDRESS=$(kubectl get svc -n kgateway-system http -o jsonpath="{.status.loadBalancer.ingress[0]['hostname','ip']}")
echo $INGRESS_GW_ADDRESS

kubectl port-forward deployment/http -n kgateway-system 8080:8080

Confirm that Solo Enterprise for kgateway correctly routes requests to Lambda by sending a curl request to the echo function. Note that the first request might take a few seconds to process, because the AWS Security Token Service (STS) credential request must be performed first. However, after the credentials are cached, subsequent requests are processed more quickly.
curl -H "Host: lambda.${AWS_LAMBDA_REGION}.amazonaws.com" \ $INGRESS_GW_ADDRESS:8080/echo \ -d '{"key1":"value1", "key2":"value2"}' -X POST
curl -H "Host: lambda.${AWS_LAMBDA_REGION}.amazonaws.com" \ localhost:8080/echo \ -d '{"key1":"value1", "key2":"value2"}' -X POST
Example response:
```
{"statusCode":200,"body":"Response from AWS Lambda. Here's the request you just sent me: {\"key1\":\"value1\",\"key2\":\"value2\"}"}% 
```

Resource-based configuration

Use AWS resource-based configuration to configure routing to functions in different accounts. The resource-based policy can give other accounts the permission to invoke the function, without requiring the other account to assume a role.

Before you begin

Follow the Get started guide to install Solo Enterprise for kgateway.
Follow the Sample app guide to create a gateway proxy with an HTTP listener and deploy the httpbin sample app.

Get the external address of the gateway and save it in an environment variable.

export INGRESS_GW_ADDRESS=$(kubectl get svc -n kgateway-system http -o jsonpath="{.status.loadBalancer.ingress[0]['hostname','ip']}")
echo $INGRESS_GW_ADDRESS

kubectl port-forward deployment/http -n kgateway-system 8080:8080

AWS resources

For the AWS configuration, you create a user or role in the authentication account, and a Lambda function in the account that contains the Lambda functions. The Lambda function has a resource-based policy statement which allows the user or role in the authentication account to invoke it.

In your authentication account, create the following resources.
1. Create a user or role in your authentication account. Be sure to give the user or role the lambda:InvokeFunction permission, so that the role can be used to invoke the Lambda functions in the other account.
2. Create an access key for the user or role, which is used to authenticate with AWS when invoking the Lambda functions.
3. Create a Kubernetes secret that contains the access key and secret key.
  kubectl apply -n kgateway-system -f - << EOF apiVersion: v1 kind: Secret metadata: name: aws-creds stringData: accessKey: ${AWS_ACCESS_KEY_ID} secretKey: ${AWS_SECRET_ACCESS_KEY} sessionToken: "" type: Opaque EOF
In your Lambda account, create the following resources.
1. Choose an existing or create a new Lambda function that you want to route to. To create a simple echo Node.js function for testing, see Create a Lambda function for testing.
2. Define a resource-based policy statement for the function, which allows the user in the authentication account to invoke it.
  1. In the AWS console, select the Lambda function.
  2. Click the Configuration tab.
  3. In the sidebar, click the Permissions tab.
  4. In the Resource-based policy statements section, click Add Permissions.
    - Select AWS account as the entity which invokes the function.
    - Specify the ARN of the user or role in the authentication account as the principal.
    - Select lambda:InvokeFunction as the action.

Set up routing to your function

Create Backend and HTTPRoute resources to route requests to the Lambda function.

Create a Backend resource that references the AWS region, the ID of the account that contains the IAM role for Lambda invocation, and the echo function that you created.

kubectl apply -f - <<EOF
apiVersion: enterprisekgateway.solo.io/v1alpha1
kind: Backend
metadata:
  name: lambda
  namespace: kgateway-system
spec:
  type: AWS
  aws:
    region: ${AWS_LAMBDA_REGION}
    accountId: "${LAMBDA_ACCOUNT_ID}"
    lambda:
      functionName: echo
EOF

Create an HTTPRoute resource that references the lambda Backend.

kubectl apply -f - <<EOF
apiVersion: gateway.networking.k8s.io/v1
kind: HTTPRoute
metadata:
  name: lambda
  namespace: kgateway-system
spec:
  parentRefs:
    - name: http
      namespace: kgateway-system
  rules:
  - matches:
    - path:
        type: PathPrefix
        value: /echo
    backendRefs:
    - name: lambda
      namespace: kgateway-system
      group: enterprisekgateway.solo.io
      kind: Backend
EOF

Get the external address of the gateway and save it in an environment variable.

export INGRESS_GW_ADDRESS=$(kubectl get svc -n kgateway-system http -o jsonpath="{.status.loadBalancer.ingress[0]['hostname','ip']}")
echo $INGRESS_GW_ADDRESS

kubectl port-forward deployment/http -n kgateway-system 8080:8080

Confirm that Solo Enterprise for kgateway correctly routes requests to Lambda by sending a curl request to the echo function. Note that the first request might take a few seconds to process, because the AWS Security Token Service (STS) credential request must be performed first. However, after the credentials are cached, subsequent requests are processed more quickly.
curl $INGRESS_GW_ADDRESS:8080/echo -d '{"key1":"value1", "key2":"value2"}' -X POST
curl localhost:8080/echo -d '{"key1":"value1", "key2":"value2"}' -X POST
Example response:
```
{"statusCode":200,"body":"Response from AWS Lambda. Here's the request you just sent me: {\"key1\":\"value1\",\"key2\":\"value2\"}"}% 
```

Cleanup

You can remove the resources that you created in this guide.

Resources for the `echo` function

Delete the lambda HTTPRoute and lambda Backend.

kubectl delete HTTPRoute lambda -n kgateway-system
kubectl delete Backend lambda -n kgateway-system

Use the AWS Lambda console to delete the echo test function.

Resource-based configuration only

If you no longer need to access Lambda functions from Solo Enterprise for kgateway, delete the aws-creds secret.

kubectl delete secret aws-creds -n kgateway-system

IRSA role-chain configuration only

If you no longer need to access Lambda functions from Solo Enterprise for kgateway:

Delete the EnterpriseKgatewayParameters resources.

kubectl delete EnterpriseKgatewayParameters http-lambda -n kgateway-system

Remove the reference to the http-lambda EnterpriseKgatewayParameters from the http Gateway.

kubectl apply -f- <<EOF
kind: Gateway
apiVersion: gateway.networking.k8s.io/v1
metadata:
  name: http
  namespace: kgateway-system
spec:
  gatewayClassName: enterprise-kgateway
  listeners:
  - protocol: HTTP
    port: 8080
    name: http
    allowedRoutes:
      namespaces:
        from: All
EOF

Delete the pod identity webhook.

kubectl delete deploy pod-identity-webhook

Remove cert-manager.

kubectl delete -f cert-manager.yaml -n cert-manager
kubectl delete ns cert-manager

Delete the AWS IAM resources from the Lambda account.

aws iam detach-role-policy --role-name lambda-invoke-role --policy-arn=arn:aws:iam::${LAMBDA_ACCOUNT_ID}:policy/lambda-invoke-policy
aws iam delete-role --role-name lambda-invoke-role
aws iam delete-policy --policy-arn=arn:aws:iam::${LAMBDA_ACCOUNT_ID}:policy/lambda-invoke-policy

Delete the AWS IAM resources from the authentication account.

aws iam detach-role-policy --role-name lambda-auth-role --policy-arn=arn:aws:iam::${AUTH_ACCOUNT_ID}:policy/lambda-auth-policy
aws iam delete-role --role-name lambda-auth-role
aws iam delete-policy --policy-arn=arn:aws:iam::${AUTH_ACCOUNT_ID}:policy/lambda-auth-policy

Access AWS Lambda with a service account Transform responses

Configure cross-account Lambda access

About

IRSA role-chained configuration

Save AWS account details

Configure AWS IAM resources

Deploy the Amazon EKS Pod Identity Webhook

Install Solo Enterprise for kgateway

Annotate the gateway proxy service account

Set up routing to your function

Resource-based configuration

Before you begin

AWS resources

Set up routing to your function

Cleanup

Resources for the echo function

Resource-based configuration only

IRSA role-chain configuration only

Resources for the `echo` function