一本讲测量误差的书的笔记，以及其中一个改变了我做实验方式的观点。
Notes from a book about measurement error, and the one idea from it that changed how I set up experiments.
一本讲测量的书的简短笔记——主要是给自己看的，因为下面这个观点今年已经改掉过我三个决定。
测量误差很少是独立的，也很少是对称的。仪器都对着同一个基准校准，观测者接受的是同一套训练，协议是同一个人写的。所以来自不同来源的误差，彼此之间的「一致」往往比偶然该有的一致更多。
这意味着那句常见的安慰——「误差会平均掉」——经常是假的。取平均能消掉随机误差，它对每一次测量都共享的那个偏差毫无作用。
重新标注多少才够，我没有好答案。我现在用的经验规则——重标 10%，看离散程度，若它比效应还大就再来一轮——是个启发式，而且我怀疑在小样本下它是个坏规则。记在这里，免得我忘了回来处理它。
Short notes on a book about measurement — mostly for myself, because the idea below has already changed three decisions this year.
Measurement errors are rarely independent and rarely symmetric. Instruments are calibrated against the same reference, observers share the same training, and protocols are written by the same person. So the errors from different sources agree with each other more often than chance would suggest.
That means the usual comfort — "errors will average out" — is often false. Averaging removes random error. It does nothing to a bias that every measurement shares.
I do not have a good answer for how much re-labelling is enough. The practical rule I use — re-label ten percent, inspect the spread, repeat if it is larger than the effect — is a heuristic, and I suspect it is a bad one at small sample sizes. Noted here so that I remember to come back to it.