labeling

Random sampling is the baseline you have to beat, and honest candidates admit it sometimes wins. Active learning earns its keep by labeling where the model is least sure, but uncertainty alone gives you a batch of near-duplicates, so you also need diversity, and a cold-start plan.

Thirty percent disagreement is not a fact about the annotators. It is a fact about the task definition, and on a skewed label set it can mean the labels are worse than chance. The week is spent on the guideline and the gold set, not on the model, because no model trains past the noise in its labels.