Building an API with FastAPI

A model in a notebook helps nobody. Behind an HTTP API any app at all can use it.

FastAPI: you declare the input and output as Pydantic models → validation, error messages and documentation (/docs) for free. Async endpoints suit LLM calls.

The minimum for something meant to run for real:

• a /healthz that answers quickly (for monitoring),
• validation of the input (length, type) → 422 automatically,
• clear errors: 400 for bad input, 503 when the model or provider is down, never a 500 with a stack trace,
• a timeout on everything external,
• logging of every call (without sensitive content),
• a limit on the number of calls per client.

from fastapi import FastAPI, HTTPException
from pydantic import BaseModel, Field
import httpx, asyncio

app = FastAPI(title="Sentiment API")

class In(BaseModel):
    text: str = Field(min_length=1, max_length=2000)

class Out(BaseModel):
    label: str
    score: float

@app.get("/healthz")
async def healthz():
    return {"ok": True}

@app.post("/v1/sentiment", response_model=Out)
async def sentiment(body: In):
    try:
        async with httpx.AsyncClient(timeout=10) as c:
            r = await c.post(MODEL_URL, json={"text": body.text})
            r.raise_for_status()
    except (httpx.TimeoutException, httpx.HTTPError):
        raise HTTPException(503, "The model is not responding right now — try again")
    d = r.json()
    return Out(label=d["label"], score=d["score"])

Run it: uvicorn app:app --port 8000. Test it with httpx.AsyncClient(app=app) in pytest — no network calls needed.