Make Your App Ready for OpenAI Models on Google Cloud

In one sitting you will know how to make your app ready to run OpenAI models on Google Cloud by changing environment variables, not code.

Why now: OpenAI has announced that its models are coming to Google Cloud. If your company already pays Google for compute, identity and billing, the model stops being a separate vendor relationship. It becomes a line item on a bill you already have.

My read: the lock-in just moved. It used to be the model. Now it is the cloud contract around it: IAM, regions, quotas, procurement. Teams that hardcoded a provider into their call sites will pay for that choice. Teams with one seam will not.

Here is the seam.

1. Put every model call behind a single function in a single file.
2. Move base URL, API key, and model name into environment variables.
3. Call through an OpenAI-compatible client, so the base URL is the only thing that changes between providers.
4. Save a golden set of prompts from your real traffic, each with the output you expect.
5. Run the golden set against your current provider and store the results.
6. When the Google Cloud endpoint reaches your project, change the variables, rerun the set, and diff.

Paste this:

import os
from openai import OpenAI

client = OpenAI(
    base_url=os.environ["LLM_BASE_URL"],
    api_key=os.environ["LLM_API_KEY"],
)
MODEL = os.environ["LLM_MODEL"]

def complete(prompt: str) -> str:
    r = client.chat.completions.create(
        model=MODEL,
        messages=[{"role": "user", "content": prompt}],
    )
    return r.choices[0].message.content

# .env (placeholders; fill in from the Google Cloud docs when available)
LLM_BASE_URL=https://YOUR-ENDPOINT/
LLM_API_KEY=YOUR-CREDENTIAL
LLM_MODEL=YOUR-MODEL-ID

The gotcha: the same model will not necessarily carry the same ID, and Google Cloud will likely want its own authentication rather than a static key. The headline does not tell you the endpoint shape, so read the docs when it lands in your project. You will recognise the problem fast. A not-found error on the model name means the ID is wrong. A call that worked in the morning and fails with an auth error later means you pasted a short-lived token where a durable credential belongs. Wrap the key lookup in a function now so you can refresh it later.

The golden set is the part people skip, and it is the part that decides whether switching is safe. Same model family, different hosting: you still verify behaviour before you move traffic. The Forge write-up shows why the harness around a model matters as much as the model.

Subscribe at /subscribe/ and the next field notes like this land in your inbox.