Authenticate with API keys

In this guide, you create an API key for authentication, and create a Gloo Gateway Upstream resource that references that API key. Then, you set up routing to that Upstream on the /openai path by creating an HTTPRoute resource. The Gloo AI Gateway automatically reads the API key from the Kubernetes secret to process authentication for requests on that path.

Set up API authentication link

1. Create an API key to access the OpenAI API. If you use another AI provider, create an API key for that provider’s AI instead, and be sure to modify the example commands in these tutorials to use your provider’s AI API instead.

2. Save the API key in an environment variable.

     export OPENAI_API_KEY=<insert your API key>
     

3. Create a Kubernetes secret to store your AI API key.

     kubectl create secret generic openai-secret -n gloo-system \
    --from-literal="Authorization=Bearer $OPENAI_API_KEY" \
    --dry-run=client -oyaml | kubectl apply -f -
     

4. Create an Upstream resource to configure an LLM provider that references the AI API key secret. To review the settings that you can specify for providers other than OpenAI, see the AI options in the Upstream reference.

     kubectl apply -f- <<EOF
   apiVersion: gloo.solo.io/v1
   kind: Upstream
   metadata:
     labels:
       app: gloo
     name: openai
     namespace: gloo-system
   spec:
     ai:
       openai:
         authToken:
           secretRef:
             name: openai-secret
             namespace: gloo-system
   EOF
     

5. Create an HTTPRoute resource that routes incoming traffic on the /openai path to the Upstream backend that you created in the previous step. In this example, the URLRewrite filter rewrites the path from /openai to the path of the API in the LLM provider that you want to use, such as /v1/chat/completions for OpenAI.

     kubectl apply -f- <<EOF
   apiVersion: gateway.networking.k8s.io/v1
   kind: HTTPRoute
   metadata:
     name: openai
     namespace: gloo-system
   spec:
     parentRefs:
       - name: ai-gateway
         namespace: gloo-system
     rules:
     - matches:
       - path:
           type: PathPrefix
           value: /openai
       filters:
       - type: URLRewrite
         urlRewrite:
           path:
             type: ReplaceFullPath
             replaceFullPath: /v1/chat/completions
       backendRefs:
       - name: openai
         namespace: gloo-system
         group: gloo.solo.io
         kind: Upstream
   EOF
     

6. Get the external address of the gateway and save it in an environment variable.

     export INGRESS_GW_ADDRESS=$(kubectl get svc -n gloo-system gloo-proxy-ai-gateway -o jsonpath="{.status.loadBalancer.ingress[0]['hostname','ip']}")
   echo $INGRESS_GW_ADDRESS  
     

     kubectl port-forward deployment/gloo-proxy-ai-gateway -n gloo-system 8080:8080
     

7. Send a request to the AI API. Verify that the request succeeds and that you get back a response from the chat completion API. Note that you do not send an API key as part of the request. Instead, the Gloo AI Gateway automatically reads the API key from the Kubernetes secret and adds it as a request header.

     curl "$INGRESS_GW_ADDRESS:8080/openai" -H content-type:application/json  -d '{
      "model": "gpt-3.5-turbo",
      "messages": [
        {
          "role": "system",
          "content": "You are a poetic assistant, skilled in explaining complex programming concepts with creative flair."
        },
        {
          "role": "user",
          "content": "Compose a poem that explains the concept of recursion in programming."
        }
      ]
    }' | jq
     

   Example output:

     {
     "id": "chatcmpl-AEHYs2B0XUlEioCduH1meERmMwBGF",
     "object": "chat.completion",
     "created": 1727967462,
     "model": "gpt-3.5-turbo-0125",
     "choices": [
       {
         "index": 0,
         "message": {
           "role": "assistant",
           "content": "In the world of code, a method elegant and rare,\nKnown as recursion, a loop beyond compare.\nLike a mirror reflecting its own reflection,\nIt calls upon itself with deep introspection.\n\nA function that calls itself with artful grace,\nDividing a problem into a smaller space.\nLike a nesting doll, layers deep and profound,\nIt solves complex tasks, looping around.\n\nWith each recursive call, a step is taken,\nTowards solving the problem, not forsaken.\nA dance of self-replication, a mesmerizing sight,\nUnraveling complexity with power and might.\n\nBut beware of infinite loops, a perilous dance,\nWithout a base case, it’s a risky chance.\nFor recursion is a waltz with a delicate balance,\nInfinite beauty, yet a risky dalliance.\n\nSo embrace the concept, in programming’s domain,\nLet recursion guide you, like a poetic refrain.\nA magical loop, a recursive song,\nIn the symphony of code, where brilliance belongs.",
           "refusal": null
         },
         "logprobs": null,
         "finish_reason": "stop"
       }
     ],
     "usage": {
       "prompt_tokens": 39,
       "completion_tokens": 200,
       "total_tokens": 239,
       "prompt_tokens_details": {
         "cached_tokens": 0
       },
       "completion_tokens_details": {
         "reasoning_tokens": 0
       }
     },
     "system_fingerprint": null
   }
     

Next link

Now that you successfully authenticated with your LLM provider, you can go ahead and set up access control and prompt guards for the LLM.