Authenticate with API keys

Set up API authentication link

1. Create an API key to access the OpenAI API and save it in an environment variable.

     export OPENAI_API_KEY=<insert your API key>
     

2. Create a Kubernetes secret to store your OpenAI API key.

     kubectl create secret generic openai-secret -n gloo-system \
    --from-literal="Authorization=Bearer $OPENAI_API_KEY" \
    --dry-run=client -oyaml | kubectl apply -f -
     

3. Create an Upstream resource that configures an LLM provider that references the API key secret that you created earlier.

     kubectl apply -f- <<EOF
   apiVersion: gloo.solo.io/v1
   kind: Upstream
   metadata:
     labels:
       app: gloo
     name: openai
     namespace: gloo-system
   spec:
     ai:
       openai:
         authToken:
           secretRef:
             name: openai-secret
             namespace: gloo-system
   EOF
     

4. Create an HTTPRoute resource that routes incoming traffic on the /openai path to the Upstream backend that you created earlier. In the example, the URLRewrite filter rewrites the path from /openai to the path of the API in the LLM provider that you want to use, /v1/chat/completions.

     kubectl apply -f- <<EOF
   apiVersion: gateway.networking.k8s.io/v1
   kind: HTTPRoute
   metadata:
     name: openai
     namespace: gloo-system
   spec:
     parentRefs:
       - name: ai-gateway
         namespace: gloo-system
     rules:
     - matches:
       - path:
           type: PathPrefix
           value: /openai
       filters:
       - type: URLRewrite
         urlRewrite:
           path:
             type: ReplaceFullPath
             replaceFullPath: /v1/chat/completions
       backendRefs:
       - name: openai
         namespace: gloo-system
         group: gloo.solo.io
         kind: Upstream
   EOF
     

5. Get the external address of the gateway and save it in an environment variable.

     export INGRESS_GW_ADDRESS=$(kubectl get svc -n gloo-system gloo-proxy-ai-gateway -o jsonpath="{.status.loadBalancer.ingress[0]['hostname','ip']}")
   echo $INGRESS_GW_ADDRESS  
     

     kubectl port-forward deployment/gloo-proxy-ai-gateway -n gloo-system 8080:8080
     

6. Send a request to the OpenAI API. Verify that the request succeeds and that you get back a response from the chat completion API. Note that you do not send an API key as part of the request. Instead, the Gloo AI Gateway automatically reads the API key from the Kubernetes secret and adds it as a request header.

     curl -v "$INGRESS_GW_ADDRESS:8080/openai" -H content-type:application/json  -d '{
      "model": "gpt-3.5-turbo",
      "messages": [
        {
          "role": "system",
          "content": "You are a poetic assistant, skilled in explaining complex programming concepts with creative flair."
        },
        {
          "role": "user",
          "content": "Compose a poem that explains the concept of recursion in programming."
        }
      ]
    }' | jq
     

   Example output:

     {
     "id": "chatcmpl-9Z3f0tJ5x4EjsKYZDsk100Q0KOR2v",
     "object": "chat.completion",
     "created": 1718142578,
     "model": "gpt-3.5-turbo-0125",
     "choices": [
       {
         "index": 0,
         "message": {
           "role": "assistant",
           "content": "In the realm of code, there lies a wondrous trick,\nA concept so elegant, it'll make your mind pick.\nRecursion it's called, a loop with a twist,\nA function that calls itself, a dance in the mist.\n\nLike a mirror reflecting its own reflection,\nRecursion calls back to solve a problem's inception.\nIt breaks tasks into smaller pieces with grace,\nSolving them one by one, in a recursive embrace.\n\nJust like a Russian nesting doll so grand,\nEach layer revealing a solution so grand.\nThe base case, the exit, the end of the line,\nCompletes the recursion, oh so divine.\n\nInfinite loops it can cause if not tamed,\nBut with careful design, it can be named.\nA powerful tool in the programmer's hand,\nRecursion, a concept so enchanting and grand."
         },
         "logprobs": null,
         "finish_reason": "stop"
       }
    ],
    "usage": {
       "prompt_tokens": 39,
       "completion_tokens": 169,
       "total_tokens": 208
     },
     "system_fingerprint": null
   }
     

Next link

Now that you successfully authenticated with your LLM provider, you can go ahead and set up access control and prompt guards for the LLM.