Model cards
← ARI3 overview
One card per ARI3 version. Every version so far is ARI3 Perception, which decides whether a news item is about style. The full experiment behind each version is in its notebook entry.
Shared architecture
| Embedder | paraphrase-multilingual-MiniLM-L12-v2, revision e8f8c21, 384 dimensions, frozen Reimers & Gurevych 2019, 2020 |
|---|---|
| Classifier | Logistic regression, L2, C = 1.0 |
| Calibration | Temperature scaling on a held-out calibration split Guo et al. 2017 |
| Uncertainty | Split conformal prediction, score 1 − p(true label), α = 0.10 Angelopoulos & Bates 2021 |
DISCIPLINA v0.0.2
CurrentNotebook entry
- Purpose
- Decide whether a news item is about style, and say when it is not sure.
- Inputs
- A news item's headline and feed excerpt, joined and cut to 1,000 characters, in any language the embedder covers.
- Outputs
- A calibrated probability that the item is about style, and a conformal set: {yes}, {no}, or {yes, no} when it is not sure (target coverage 90%).
- Training data
- 229 labels by the editor: 194 random and 35 chosen by active learning. Calibrated on 45 random labels. Tested on 61 random labels. Architecture unchanged from v0.0.1.
- Intended use
- Filtering collected items before counting, with not-sure items meant for the editor.
- Not for
- Judging individual articles or outlets, or any use outside the index.
Metrics
| Accuracy (61 held-out items) | 0.902, 95% CI 0.80–0.95 |
|---|---|
| Calibration error after scaling | 0.025 |
| Conformal coverage | 0.967 |
| Confident (single-answer) share | 0.754 |
| Confident mistakes | 2 of 61 |
Known limitations
- Trained and tested on one editor's labels.
- Mostly English-tagged training items. Other languages are barely tested.
- Reads headlines and excerpts only, never full articles.
- Most labeled items were published in September 2026.
Failure modes
- Articles that sit next to style, such as beauty or retail business news, are where it errs most.
- More decisive than v0.0.1, and it made 2 confident mistakes where v0.0.1 made none.
INDUSTRIA v0.0.1
Superseded by v0.0.2, kept frozenNotebook entry
- Purpose
- Decide whether a news item is about style, and say when it is not sure.
- Inputs
- A news item's headline and feed excerpt, joined and cut to 1,000 characters.
- Outputs
- A calibrated probability and a conformal set at 90% target coverage.
- Training data
- 82 random labels by the editor. Calibrated on 29. Tested on 34.
- Intended use
- The baseline for later versions.
- Not for
- Current use. v0.0.2 replaces it.
Metrics
| Accuracy (34 held-out items) | 0.882, 95% CI 0.73–0.95 |
|---|---|
| Calibration error after scaling | 0.088 |
| Conformal coverage | 1.000 |
| Confident (single-answer) share | 0.559 |
| Confident mistakes | 0 of 34 |
Known limitations
- Very small training, calibration and test sets.
- Cautious: most unlabeled items received the not-sure answer.
Failure modes
- Half its test mistakes were beauty trade coverage, before the beauty scope rule existed.
Version history
- v0.0.2 DISCIPLINA: current.
- v0.0.1 INDUSTRIA: superseded by v0.0.2, kept frozen.