EXPLAINER

Model APIs for Applications

A model API is an HTTP service that accepts messages and returns generated text or tool calls. You choose a model, send a prompt with authentication, handle tokens and errors, and keep the vendor SDK or raw HTTP behind a small interface you own.

Short answer: A model API is HTTP: messages in, text or tool calls out, billed per token. Call it from your server, not from a leaked browser key. SDKs help; they are not the architecture. For open weights on your own machine see Local LLM Runtimes.

What it is

A model API is a network service that runs a model for you. The usual shape, popularised by chat completions, is a POST with a model name, a list of messages (system, user, assistant), and parameters such as temperature and a max token limit. The response contains generated text, a finish reason, and usage counts. Variants add tool calling, image inputs, or streaming events. Many vendors now expose an "OpenAI-compatible" path so the same client can change the base URL.

The API is not the model. The same brand name can point at several snapshots. It is not an agent. It does not remember your user between requests unless you send the history. Hosted APIs differ from local runtimes, which serve a similar HTTP shape on a machine you control.

Why and when it is used

Use a hosted model API when you want quality and speed without operating GPUs, when traffic is spiky, or when you are still finding the product. Use a local or privately hosted model when data cannot leave your network, when unit cost at steady volume beats the token price, or when you need a fixed snapshot for reproducibility.

Do not call a model API from a mobile or browser build with a secret key embedded. Do not send regulated data until the contract and region are acceptable. Do not build your whole product as one giant prompt if a database query would answer the question. The API is a component.

How it works

Your server authenticates, usually with an API key or a cloud IAM token, and sends JSON. The provider checks quota, runs the model, and returns JSON or a stream of server-sent events. Input and output are both metered as tokens. A context-window error means you sent more than the model accepts. A 429 means you are over the rate limit. A 500 is theirs; a timeout might be a long generation you should retry idempotently.

Tool calling returns a structured request instead of user-facing text. Your code runs the tool and sends another request with the tool result. Streaming changes the client (you read chunks) but not the security model. Idempotency keys, where the provider supports them, matter if a retry would double-charge an action in your database. The model call itself is not a transaction.

Technologies and dependencies

Any language that can send HTTPS works. Python and JavaScript have the most examples. Vendor SDKs handle retries and types. A thin wrapper of your own should hide the vendor so a test can return canned text. You depend on the provider's status page, their data-retention setting, and the specific model id. Environment variables are the right place for keys. LangChain and similar frameworks sit on top and are optional.

How to get started

  1. Create a key with a spend limit. Put it in an environment variable on a server, not in the front-end repo.
  2. Send one hard-coded prompt with a small max token value. Print the raw JSON.
  3. Add a timeout. The default HTTP timeout in some stacks is infinite.
  4. Log the model id, latency and token usage. Do not log the full prompt if it contains customer data; log a length and a request id.
  5. Try a failure on purpose: an oversized prompt, a bad key. Map those errors to messages a user can understand.
  6. If you need structured output, validate it (How to Validate JSON) before it touches your database.
  7. Write two tests with recorded responses so your CI does not call the live model for every commit.

When you add a second provider, wrap both behind one function your application calls. Do not sprinkle vendor types through the UI.

Cautions and trade-offs

Model ids change. Pin a snapshot if the vendor offers one, and re-evaluate when you move. Temperature and sampling changes make tests flake if you expect exact strings; assert on structure and facts, not on one poetic paragraph. Streaming breaks naive "read the body" code.

Keys leak through client apps, logs and screenshots. Rotate them. Regional endpoints exist for a reason; sending EU personal data to an arbitrary region is a compliance bug. Retries without backoff amplify outages. And a cheaper model that fails your evaluation is not cheaper once humans fix the output. Keep a small offline set of prompts and expected facts, and run it when you change model or prompt.