Keeping up with the research front

Hundreds of ML papers are published every day. «Following the field» is therefore not reading everything, but having a filter.

A routine that lasts:

The source	The frequency	What you do
arXiv alerts (cs.CL, cs.LG) on keywords	daily, 5 min	read the titles, save 1–3
The conferences' accepted papers (NeurIPS, ICML, ACL)	per conference	skim the list, pick 5–10
A handful of chosen people and labs	continuously	they do the selecting for you
Model cards and release notes	at a release	the things that actually affect your product
Reproduction reports, negative results	monthly	this is where what did not hold is written down

The rule: read the abstract of many, the method of few, reproduce one a quarter. The last of those is what actually builds understanding.

A hype filter — six questions that sift quickly:

1. Is it compared against a tuned baseline? An untuned baseline = the result says nothing.
2. How many seeds, and is the spread reported? One seed = an anecdote.
3. Could the benchmark be contaminated? Published before the model's training data?
4. Is there code and an exact configuration? If not: treat it as a hypothesis.
5. Is the improvement larger than the noise between seeds?
6. Who has replicated it? A paper is a hypothesis; two independent replications are a result.

Signals that often mean little: a new SOTA by a few tenths, demonstrations without a quantitative evaluation, «emergent» behaviour measured with a threshold metric, and results that exist only in a blog post.

Signals that often mean a lot: methods that simplify (fewer parts, not more), results that hold across several datasets and scales, negative results that disprove something established, and tools other people quickly start using.

A practical rhythm: 20 minutes a day on titles, two hours a week on one paper in three passes, one quarter per reproduction. Write reading cards (the node läsa-papers) and save them searchably — otherwise everything is forgotten.