Project E: an NLP system end to end

Project E: a complete, runnable NLP system. A suggestion: classify incoming support tickets (category plus urgency) and expose it as an API.

Deliverables:

1. Data: ≥ 800 labelled examples (your own, synthetic with review, or an open dataset), a guideline, κ on a sample, a stratified train/val/test split.
2. Baselines: majority class plus TF-IDF/LR, tuned.
3. Model: a fine-tuned encoder; macro-F1 with a bootstrap CI; a confusion matrix; an error analysis of 30 cases.
4. Service: FastAPI with validation, /healthz, a 503 degradation (fall back to the TF-IDF model when the transformer is missing!), logging without the text.
5. CI: tests plus the eval harness with a threshold and a regression list.
6. Report: the hypothesis, the method, the results with uncertainty, the limitations, the reproducibility information (commit, data hash, seeds).

An assessment matrix (apply it to yourself before you hand in):

Part	Pass	Strong
Data	a split without leakage	κ reported, edge cases in the guideline
Baseline	exists	tuned, with a CI
Model	beats the baseline	beats it outside the CI, the error analysis leads to action
API	answers correctly	validation, the 503 fallback tested
CI	tests green	an eval threshold plus regressions block the merge
Report	complete	negative results and limitations reported

The most common shortcoming: the report has no uncertainty and the baseline is untuned. The second most common: the fallback exists in the code but has never been run.