Skip to content
Latest (currently 2026.10.2) has the newest features, bug fixes, and CVE patches of Solo Enterprise for agentgateway.

For the complete documentation index, see llms.txt. Markdown versions of all docs pages are available by appending .md to any docs URL.

OpenAI SDK

Verified : Automated tests passed for: End-to-end run: Every step ran in a fresh test environment and the results were checked.
Page as Markdown

Use the OpenAI Python and Node.js SDKs with agentgateway running in Kubernetes

Use the OpenAI Python or Node.js SDK to send requests through agentgateway deployed in Kubernetes.

Before you begin

  1. Set up an agentgateway proxy.
  2. Set up access to the OpenAI LLM provider.

Get the gateway URL

Tip

Kind cluster? Kind does not support LoadBalancer services by default. To use this option with a Kind cluster, install and run cloud-provider-kind.

The following command reads the LoadBalancer IP address or hostname, whichever your cloud provider assigns.

export INGRESS_GW_ADDRESS=$(kubectl get svc -n agentgateway-system agentgateway-proxy \
  -o jsonpath="{.status.loadBalancer.ingress[0]['hostname','ip']}")

echo "Gateway address: $INGRESS_GW_ADDRESS"

After port-forwarding, the gateway is accessible at http://localhost:8080. Use localhost:8080 wherever the instructions reference $INGRESS_GW_ADDRESS.

kubectl port-forward -n agentgateway-system svc/agentgateway-proxy 8080:80

Python

  1. Install the OpenAI SDK in your Python project.

    pip install openai
  2. Create and run the following script to send a request through agentgateway. Replace <route-path> with the path from your HTTPRoute configuration (for example, /openai).

    Note

    Do not include /v1 in the base_url — the OpenAI SDK appends it automatically.

    import os
    from openai import OpenAI
    
    gateway_address = os.environ["INGRESS_GW_ADDRESS"]
    
    client = OpenAI(
        base_url=f"http://{gateway_address}/<route-path>",
        api_key="anything",  # placeholder if gateway has no auth
    )
    
    response = client.chat.completions.create(
        model="gpt-4o-mini",
        messages=[{"role": "user", "content": "Hello from Kubernetes!"}],
    )
    print(response.choices[0].message.content)

Node.js

  1. Install the OpenAI SDK in your Node.js project.

    npm install openai
  2. Create and run the following script to send a request through agentgateway. Replace <route-path> with the path from your HTTPRoute configuration (for example, /openai).

    Note

    Do not include /v1 in the baseURL — the OpenAI SDK appends it automatically.

    import OpenAI from "openai";
    
    const gatewayAddress = process.env.INGRESS_GW_ADDRESS;
    
    const client = new OpenAI({
      baseURL: `http://${gatewayAddress}/<route-path>`,
      apiKey: "anything",
    });
    
    const response = await client.chat.completions.create({
      model: "gpt-4o-mini",
      messages: [{ role: "user", content: "Hello from Kubernetes!" }],
    });
    console.log(response.choices[0].message.content);