Specifications for metrology equipment are usually published in nice glossy brochures that have lots of great pictures highlighting all the features of the product. But specifications can be confusing and are often a hot topic of debate.

One of the good things about published specifications is in the fine print—the part that says the manufacturer has the right to change the specifications at any time. Why is it good that manufacturers can change their specs? Because, as we have discussed many times, there is an ever-increasing demand on machining to produce parts with tighter tolerances.

Gages are frequently redesigned during their lifetimes to offer improved performance with better mechanical components, better displacement sensing or even software compensation to correct repeatable errors.

With long-range measuring instruments, the accuracy specification is usually related to the range of measurement. The farther the gage moves from its starting or reference point, the more error one is apt to see. So, measuring error for a long-range linear measuring device might be specified something like: Measuring Error ≤ (2 + (L/600)) micrometers where the length of the measuring instrument is 600 mm.

The way to read this is as follows: The measuring error is equal to or less than 2 micrometers plus the result of the length displaced (L, given in millimeters) divided by 600, expressed in micrometers. At full displacement, this part of the equation is: 600 mm displaced/600 mm full range, which equals 1. Thus, at full range the equation for measuring error becomes: **Measuring error (at full range) ≤ (2 + (600/600)) = (2 + (1)) = 3 micrometers.**

If the displaced measuring location was at half range, then the equation would be: **Measuring error = (2 + (300/600)) = (2 + (0.5)) = 2.5 micrometers.**

So, we can now expect that the smallest error is apt to be 2 micrometers, while at the extreme end of the scale it can be 3 micrometers.

The certification lists the points measured and then the maximum permissible error at each point. The certificate will look like this:

Measuring
Position |
Permissable
Error |
Recorded
Error |

100 mm | 2.16µm | 1.5µm |

300 mm | 2.5µm | -1.7µm |

600 mm | 3.0µm | 2.1µm |

So far, we haven’t mentioned the negative sign or negative values. A negative value for error does not mean better than perfect. Rather, these values are part of the language of metrology. If you don’t know the language, there is a good chance you could get confused.

There is a document on the language of metrology called the “International Vocabulary of Basic and General Terms in Metrology.” One of the terms described in the Vocabulary is Maximum Permissible Error (MPE). In full, it is called the maximum permissible error of a measuring instrument, and it can also be described as the limits of permissible error (of a measuring instrument). It is defined as “the extreme values of an error permitted by the specifications, regulations, and so on, for a given measuring instrument.”

Therefore, the permissible error listed on the calibration certificate really means maximum permissible error.Because one can err in two directions with length measurement, this is reflected with both a plus and a minus value. While there is the possibility of a “0-3.0 micrometers” error for the 600 mm position, the specification means that any value between -3.0 micrometers and 3.0 micrometers is acceptable.

Here’s another example of the language of metrology: **Uncertainty = K = 0.5 micrometer** (where 0.5 micrometer can be any uncertainty as determined by the lab).

Here again, uncertainty is not a single-sided number. Rather, it is a number applied to both sides of the measured result to create a window where the true result is apt to fall. The uncertainty number does not get added to the permissible error.

Understanding this language, a purchaser of linear measuring equipment can understand the specifications and certification that come with the product. While it may be a negative, it’s really a good thing.