Same Problem, Different Resolution

In a shopfloor measurement of a machined part, the combined variation of the measurement instrument and the operator performing the measurement can leave little room for error—even when the tolerance of the feature is not extremely tight. A part measuring anywhere near the tolerance limit might be rejected as non-conforming. Though a later inspection system may determine the part to be acceptable, sending too many rejected parts on for this extra inspection is wasteful.

To solve this problem, someone usually suggests changing to a finer-resolution gage. After all, many rejected parts measure within just one increment of the tolerance limit. A finer resolution should lessen the frequency of these borderline parts.

But making that change may introduce a new problem. The shop that replaces a 0.01-mm indicator with a 0.001-mm indicator is asking its operators to master the gage more often. While a 0.01-mm indicator might never seem to lose its zero, a 0.001-mm indicator might seem unable to hold its zero. And keep in mind that the 0.01-mm indicator was always adequate outside the borderline region.

Some operators keep the old gage, but try to interpret a finer resolution. That is, they count “partial increments.” This approach is even less acceptable, because a fundamental rule of metrology states that an instrument can never perform better than its resolution. If the resolution of a dial indicator is 0.01 and we measure the part to be +0.05, then the actual value could be +0.04 or +0.06. We should never use this indicator to try to declare a value such as +0.045.

The true solution is compromise. Keep both indicators ready. Use the 0.01 mm indicator for all measurements. Then, any time a part measures within ±0.01 mm (one increment) from the tolerance limit, consider that part “too close to call” using that indicator. Confirm the master with the finer indicator, and then use this indicator to make the final judgment.

About the author: Richard Clark is a metrologist and measurement engineer in Portland, Indiana. His e-mail address is rcmetrology@yahoo.com.