Calling AI and Model APIs with Python

What is it?

Most hosted models expose a chat or completions HTTP endpoint. Your Python program sends JSON (model id, messages, optional parameters) and receives JSON containing generated text. Official SDKs are optional wrappers; understanding the raw request helps you debug any language or library.

Why it matters

A reliable call is the foundation for RAG, agents and evals. If keys leak, timeouts hang workers, or errors are ignored, every higher layer fails. Learning one thin client also makes it obvious what a framework is doing for you (AI Framework vs Direct Model API).

How to do it

1. Create an API key in the provider console; store it as MODEL_API_KEY (or the name they document).
2. Build a JSON body with the model id and a messages array (system + user roles as required).
3. Send POST with Authorization: Bearer … and Content-Type: application/json.
4. Set a timeout (tens of seconds is common for chat).
5. If the status is not 2xx, read the error body and raise or retry with backoff for transient codes.
6. Parse JSON and read the assistant text from the documented path (often choices[0].message.content — confirm for your provider).
7. Log latency and token usage fields when present, without logging full prompts that contain secrets.

Use the standard library (urllib) or an HTTP client library — both are fine. Prefer environment variables over .env files committed to git (API Authentication: Keys, Sessions and OAuth).

Example

import os, json, urllib.error, urllib.request

URL = "https://api.example.com/v1/chat/completions"

def chat(messages, model="your-model-id", timeout=60):
    key = os.environ.get("MODEL_API_KEY")
    if not key:
        raise RuntimeError("Set MODEL_API_KEY in the environment")
    payload = json.dumps({"model": model, "messages": messages}).encode()
    req = urllib.request.Request(
        URL, data=payload, method="POST",
        headers={"Authorization": f"Bearer {key}", "Content-Type": "application/json"},
    )
    try:
        with urllib.request.urlopen(req, timeout=timeout) as resp:
            data = json.load(resp)
    except urllib.error.HTTPError as e:
        detail = e.read().decode("utf-8", "replace")
        raise RuntimeError(f"Model API {e.code}: {detail}") from e
    return data["choices"][0]["message"]["content"]

text = chat([
    {"role": "system", "content": "Reply in one short paragraph."},
    {"role": "user", "content": "What is a context window?"},
])
print(text)

Adjust URL and JSON paths to your provider. For streaming, see the ideas on Streaming AI Responses for Chat UIs and implement the provider's event format carefully.

When to use it

Use a direct Python HTTP call for scripts, prototypes, CI evals and small services. Add an official SDK when you need typed helpers and the team agrees on that dependency. Move secret handling behind your own backend if a browser or mobile app is involved (JavaScript for AI Applications).

Common mistakes

• Hard-coding keys in source or notebooks.
• No timeout — hung requests pile up.
• Swallowing non-2xx responses and treating empty text as success.
• Logging raw requests that include customer data (AI Privacy and Security: What to Know Before You Paste Data).
• Mixing embedding endpoints with chat endpoints by accident.

Related concepts