Node.js for AI Applications

What is it?

JavaScript on the server — Node.js — is where many AI web apps place vendor credentials, rate limits and logging. The browser calls your Node route; Node calls the model. That split is the core architecture.

Why it matters

If the browser held the vendor key, anyone could steal it. Node (or another backend) also gives you one place to enforce auth, trim history, and attach tools (Agent Tools: How AI Agents Take Actions).

How to do it

1. Create a small HTTP server (built-in node:http, Express, Fastify, etc.).
2. Read MODEL_API_KEY from the environment at process start.
3. Add POST /api/chat that:

- authenticates the session; - validates JSON body; - calls the vendor (Calling AI Services from JavaScript and TypeScript); - returns { text } or streams (Streaming AI Responses with Node.js).

1. Set request timeouts and limit body size.
2. Log metrics without storing raw sensitive prompts unless policy allows (AI Privacy and Security: What to Know Before You Paste Data).

Example

import http from "node:http";
import { chat } from "./model.js"; // wraps vendor fetch

const server = http.createServer(async (req, res) => {
  if (req.method === "POST" && req.url === "/api/chat") {
    const chunks = [];
    for await (const c of req) chunks.push(c);
    let body;
    try { body = JSON.parse(Buffer.concat(chunks).toString("utf8")); }
    catch { res.writeHead(400); res.end('{"error":"bad json"}'); return; }
    try {
      const text = await chat([{ role: "user", content: String(body.message || "") }]);
      res.writeHead(200, { "Content-Type": "application/json" });
      res.end(JSON.stringify({ text }));
    } catch (e) {
      res.writeHead(502, { "Content-Type": "application/json" });
      res.end(JSON.stringify({ error: "upstream_failed" }));
    }
    return;
  }
  res.writeHead(404); res.end();
});
server.listen(3000);

When to use it

Use Node when your UI is JavaScript/React and you want one language across client and server. Use Python for the model proxy if the rest of your ML stack is Python — the pattern is the same.

Practical notes

Operational basics: run Node behind a reverse proxy that terminates TLS, set 'NODE_ENV=production', and limit body size (for example 1mb) unless you accept uploads elsewhere. Use a process manager that restarts on crash. Separate config for model base URL and model id so you can switch providers without code edits. Add a kill switch environment flag that returns a friendly "AI temporarily unavailable" without calling the vendor. When attaching tools, execute them in the same request lifecycle with hard timeouts. Prefer structured logs (JSON) with user id hashes rather than raw emails. This service is also where you enforce org quotas before spending tokens.

Common mistakes

• Committing .env with live keys.
• Open CORS to * on authenticated AI routes.
• No upstream timeout.
• Trusting body.message length without limits (cost attacks).

Related concepts