Five Readings of One Part

In a college workshop Dana is measuring a small steel part with a caliper. She needs its diameter: the part must fit a hole, and even a tenth of a millimetre may be one too many. The instrument reads 10.0 mm. Dana lifts it off and measures again. Now 10.2. The third time it comes out 9.8, the fourth 10.1, the fifth 9.9.

One part, and already five numbers.

“So which one is right?” Dana asks.

“Take the average,” Timur suggests. “Add them up and divide by five.”

The sum is 50.0 mm. Divide by five and you get exactly 10.0 mm. Timur is pleased: five different readings reduced to one round number. That is the one begging to be written into the logbook.

Before reading on, decide what you would write down: one number, all five, or something else.

What the average hides

Timur’s suggestion makes sense. The average takes all five readings into account and gives a handy starting point for estimating the diameter. But before accepting it as the measurement result, look at what was lost in the adding and dividing.

Picture two more notebooks. In one, 10.0 mm stands written five times. In the other, the entries run 9.5, 10.5, 9.6, 10.4, 10.0 mm. The average is the same in all three notebooks. But the readings differ: in one series they agreed, in another they spread over a whole millimetre. Behind the entry “10.0 mm” that difference is invisible.

If you chose to keep all five numbers, you can always go back and check the calculation. That is useful. But comparing results also needs a short entry that shows the spread at a glance. The average alone is too little for that.

Let us find a number that shows how wide the readings spread. And then work out what it tells us about the average itself.

How far the readings spread

Subtract the average from each reading. The differences are called deviations.

Add the deviations and you get zero. Positive and negative values cancel each other — as they will for any set of readings. So to estimate the spread we take the squared deviations instead: they are never negative and cannot cancel out in a sum. The last column totals 0.10 mm².

From here we follow the method of the international Guide to the Expression of Uncertainty in Measurement, known by its short name GUM. The sum of squares is divided by the number of readings minus one. In our case, by four.

Where did that one go? We computed the average from the same five readings, so their deviations are tied together. If four of them are known, the fifth is fixed: the sum must come out zero. The divisor “five minus one” accounts for that tie when estimating the spread.

We get 0.10 / 4 = 0.025 mm². That is the sample variance. Take the square root to express the spread in millimetres: about 0.16 mm. This quantity is called the standard deviation.

It characterises the spread of readings in our series. It is handy for comparing series: the same calculation for the notebook with readings from 9.5 to 10.5 mm gives about 0.45 mm. Same average, but a standard deviation nearly three times larger. Now two numbers show the difference.

What we learned about the average

Dana writes the average of five readings into the logbook. If five more measurements are made, the numbers will most likely come out slightly different. The average will move too. How far can it wander from series to series?

The answer depends on how we measure. Suppose it is the same diameter, the measurement conditions do not change, and the random wobbles of the readings are independent of each other. Then averaging damps down those wobbles. In a real workshop these assumptions would have to be justified by looking into how the measurements were actually made — the five numbers alone give no such guarantee.

Next, divide the variance we found by the number of readings: 0.025 / 5 = 0.005 mm². Its root is about 0.07 mm. That is the estimated standard uncertainty of the average from the spread of readings. It characterises how the average might wobble when such series are repeated.

The two numbers have different jobs. The standard deviation of 0.16 mm describes the spread of single readings. The 0.07 mm helps estimate how that spread carries over to the average of five. This way of evaluating uncertainty — from a run of observations — is called Type A in the Guide.

Dana adds a line: “Five readings; the average is 10.0 mm; the standard uncertainty of the average from the spread is about 0.07 mm.”

The word “uncertainty” here is easy to misread as an admission of error. How far the result lies from the exact diameter, we do not know yet. We estimated how random spread affects the averaged result. This calculation will not catch a shift shared by all readings: if the instrument keeps overstating by the same amount, averaging preserves the shift.

What five numbers do not say

At the start of the story Dana needed to know whether the part would fit the hole. Did the computed average bring us closer to an answer?

Closer — but a decision still takes work with the instrument. Suppose its scale reads in 0.1 mm steps. On such a scale close values inside one division cannot be told apart. Even five identical entries would not mean the diameter is known exactly. We also need to find out whether the instrument was checked and whether its readings need a correction.

The part itself raises questions too. Where did Dana put the caliper’s jaws? What if the cross-section is slightly oval? Then she measured different sizes in different directions. Before averaging them, we must decide which size we are after. For fitting a hole, one average may not be enough.

To estimate the other components of uncertainty, we can turn to the instrument’s data sheet or its calibration certificate. Evaluation from such information is called Type B evaluation. The letters A and B mark ways of evaluating: they differ in where the information comes from and how it is handled. Some influences may already show in the spread of readings. In the combined evaluation each must be counted only once.

For now our 0.07 mm covers only this series. It is tempting to shorten the entry to “10.0 ± 0.07 mm” and decide the diameter must lie between 9.93 and 10.07 mm. But the calculation gives no such guarantee. Even a complete standard-uncertainty evaluation by itself sets no boundary the value cannot cross.

Now Dana can say more precisely what she knows. The average of five readings is 10.0 mm. Their standard deviation is about 0.16 mm. Under the assumed conditions, the standard uncertainty of the average evaluated from the spread is about 0.07 mm. The calculation can be repeated from the notebook. But to check its conditions and draw conclusions about the diameter, we will have to go back to the instrument and the part.

The round number stayed the same. But now we understand how it was obtained, what stands behind it, and what is still missing for the task. An estimate can be defended even when the exact value is unknown.

In the next chapter Dana will receive a laboratory report on measuring the same part. It will hold a number with many digits after the point, another instrument’s name, and a specialist’s signature. Will that mean more is known?