Most annotation problems surface later as model-quality problems, even though the root cause is usually inconsistent labeling policy.
Three rules for label consistency
- Document ambiguous cases explicitly.
- Audit inter-annotator disagreement early.
- Version guideline changes over time.