What is it?
A model predicts tokens. An HTTP API lets you send prompts and receive text or tool calls. An AI framework adds higher-level building blocks: prompt templates, tool registries, retrieval helpers, multi-step "chains" or graphs, tracing, and sometimes multi-agent wiring. Examples people cite (as categories, not endorsements) include LangChain, LlamaIndex, CrewAI, AutoGen, Semantic Kernel and the Vercel AI SDK.
Why it matters
Frameworks can speed up glue code — or hide important behaviour behind magic. Knowing what layer you are on prevents confusion like "the framework hallucinated" when the model did, or "we need a framework" when a forty-line script would do (AI Framework vs Direct Model API).
How to do it
Think in layers:
- Model + API credentials.
- Your thin client (Calling AI and Model APIs with Python / Calling AI Services from JavaScript and TypeScript).
- Optional framework for orchestration.
- Your product rules, auth and UI.
Before adopting a framework, list the jobs you need (tools, RAG, retries, streaming UI). If the list is short, stay direct. If you are reinventing graphs and tool routing for the third time, evaluate frameworks (How to Choose an Agent Framework).
Example
Direct API: your code sends messages, parses a tool call, runs SQL, returns the result.
Framework-assisted: you register tools once, the library loops model→tool→model until an end condition, and may log traces. Same underlying Function Calling (Tool Calling) Explained; different amount of shared infrastructure.
When to use it
Consider a framework for multi-step agents, shared tool ecosystems, or standardised RAG pipelines across a large team. Skip it for single-prompt features, CRON summaries, or learning how models work.
Practical notes
Frameworks also differ in how opinionated they are about prompts, memory and observability. Some push you toward a particular vector store or cloud; others stay closer to thin helpers. Read the dependency tree: a "simple" install may pull many packages you must secure and license-check. For regulated environments, prefer code you can audit line by line — which often means direct API calls plus a few internal modules. Teaching materials frequently start with frameworks for speed; when you teach foundations, start with What is a Large Language Model (LLM)?, Function Calling (Tool Calling) Explained and What is RAG (Retrieval-Augmented Generation)? first so the framework is demystified. See How to Choose an Agent Framework when you are ready to pick a category. Work through one real request end to end, write down the failure modes you observed, and add a regression test so the next change does not silently drop validation, auth, or logging. Prefer boring, reviewable code over a clever abstraction until the second or third product needs the same pattern. Work through one real request end to end, write down the failure modes you observed, and add a regression test so the next change does not silently drop validation, auth, or logging. Prefer boring, reviewable code over a clever abstraction until the second or third product needs the same pattern.
Common mistakes
- Starting with a framework tutorial before a raw API hello-world.
- Assuming the framework provides security boundaries — it usually does not.
- Coupling business logic so deeply you cannot swap vendors.