Measurement method

From CEOpedia
Measurement method
See also


Measurement method - is, in the words of the International Vocabulary of Metrology (VIM), a "generic description of a logical organization of operations used in a measurement"[1]. It answers the question how, in principle, is the comparison going to be made? Will the quantity be read straight off an instrument, calculated from other quantities, balanced against a known standard, or swapped for one?

The definition is deliberately abstract. A method is neither an instrument nor a step-by-step instruction sheet; it sits between the physical phenomenon that makes measurement possible and the procedure an operator follows on Monday morning. Get the method wrong and no careful procedure will rescue the result. That's why laboratories, quality departments and researchers spend so much effort choosing, validating and comparing methods.

Definition and place in the VIM hierarchy

The VIM (JCGM 200:2012) builds a short chain of related concepts. The measurement principle (definition 2.4) is the "phenomenon serving as a basis of a measurement", for example the thermoelectric effect applied to the measurement of temperature. The measurement method (2.5) organises operations around that principle. The measurement procedure (2.6) is the "detailed description of a measurement according to one or more measurement principles and to a given measurement method, based on a measurement model and including any calculation to obtain a measurement result". A note adds that a procedure is usually documented in enough detail for an operator to perform the measurement and is sometimes called a standard operating procedure (SOP)[1]. Finally, the measuring system (3.2) is the hardware: one or more measuring instruments and often other devices, including reagents and supplies, assembled to give measured quantity values.

Level VIM number What it describes Example: weighing Example: temperature
Measurement principle 2.4 The phenomenon that is exploited Gravitational force acting on a mass Thermoelectric effect (voltage between two dissimilar metals)
Measurement method 2.5 Generic logic of the operations Null method on an equal-arm balance, or substitution weighing Differential: the thermocouple responds to the difference between the measuring junction and a reference junction
Measurement procedure 2.6 Detailed, documented steps and calculations Warm-up time, check with a reference weight, tare, three readings, mean, rounding rule Immersion depth, stabilisation time, cold-junction compensation, conversion table used
Measuring system 3.2 The physical set-up Balance, set of reference weights, draft shield Thermocouple, extension cable, readout unit

The wording has shifted over the years. The 1993 edition of the VIM, quoted in the GUM, called a method of measurement a "logical sequence of operations, described generically" and defined a procedure as a "set of operations, described specifically"[2]. The idea stayed the same. Generic versus specific. Method versus recipe.

One more detail from the VIM matters for anyone outside the physics lab. Its note to definition 2.1 says that measurement "does not apply to nominal properties", and that measurement implies comparison of quantities or counting of entities[1]. Sorting complaints into "delivery" and "billing" is therefore not measurement in strict metrological language. Social scientists use the word more loosely.

Classification of measurement methods

The VIM note to definition 2.5 lists two ways of qualifying methods: substitution, differential and null methods on one side, direct and indirect methods on the other[1]. The two lists overlap. A null method can be direct; an indirect method can use substitution for one of its inputs.

Direct method

The value of the measurand is obtained straight from the instrument, without calculating it from other measured quantities. A tape measure on a pallet. A digital balance showing 24.7 kg. A stopwatch timing a changeover. Corrections may still be applied (for temperature, say), but the measurand isn't computed from a model linking it to different quantities.

Indirect method

Here the result comes from other quantities through a known relation. The GUM puts it plainly: "in most cases, a measurand Y is not measured directly, but is determined from N other quantities X1, X2, ..., XN through a functional relationship f"[3]. Density is the textbook case. Nobody owns a density ruler; you measure mass and volume and divide:

\[ \rho = \frac{m}{V} \]

A block with a mass of 540 g and a volume of 200 cm³ has a density of 2.7 g/cm³. The catch? Errors in both inputs feed into the result, so indirect methods need a measurement model and an uncertainty budget. Much management measurement is indirect too: labour productivity, inventory turnover, overall equipment effectiveness.

Substitution method

The measurand is replaced by a known quantity of the same kind, chosen or adjusted so that the indicating device shows the same reading. The classic case is substitution weighing on a two-pan balance (often credited to Borda): the object is balanced with a tare, removed, and replaced by standard weights until balance returns. Unequal arm lengths no longer matter, because the object and the standards sat in the same pan. The instrument acts only as a comparator.

Differential method

Only the small difference between the measurand and a known reference of nearly the same value is measured. A dial comparator zeroed on a 25 mm gauge block and then used on a machined pin is a good example. If the pin reads +0.012 mm, its diameter is 25.012 mm, and the comparator only needed to be accurate over a tiny range. Thermocouples work this way by nature: the voltage depends on the temperature difference between two junctions.

Null (zero) method

A known quantity is adjusted until its effect exactly cancels the effect of the measurand, and a detector shows zero. The equal-arm balance is one example. The Wheatstone bridge is another. Four resistors form a bridge and a galvanometer sits across the middle; when it reads zero, the ratios are equal and the unknown resistance follows from the three known ones:

\[ R_x = \frac{R_2 \cdot R_3}{R_1} \]

The beauty of a null method is that the detector doesn't have to be accurate. It only has to be sensitive enough to tell zero from not-zero.

Other ways of classifying methods

In practice engineers and quality staff use several further distinctions:

  • Comparison methods - strictly, every measurement is a comparison, but the term usually means direct comparison with a physical standard (a gauge, a reference sample, a colour card).
  • Contact and non-contact methods - a micrometer touches the part; a laser scanner, an infrared thermometer or a vision system doesn't. Non-contact methods suit soft, hot, moving or fragile objects, though they bring their own error sources such as surface reflectivity.
  • Destructive and non-destructive methods - tensile testing, oven drying and crash testing consume the sample. Ultrasonic, radiographic and eddy-current techniques leave it usable (see non destructive testing). Destructive methods often serve as the reference against which faster non-destructive ones are checked.

Measurement methods in social and management research

Outside the laboratory, "method" is used more freely, but the same logic applies. Stanley Smith Stevens, in his 1946 paper in Science, described measurement as "the assignment of numerals to objects or events according to rules" and proposed four scales: nominal, ordinal, interval and ratio[4]. Each scale is defined by the transformations that leave it unchanged, and each allows a different set of statistics. Counting cases is fine on a nominal scale; a median makes sense on an ordinal one; means and standard deviations need at least an interval scale; ratios ("twice as much") need a true zero. The VIM, note, refuses to call nominal classification measurement at all.

Typical measurement methods in management and social research include:

  • Questionnaires and rating scales - self-reports of attitudes, satisfaction, engagement. Cheap and scalable. Prone to response styles and social desirability effects, covered under measurement error.
  • Structured observation - trained observers code behaviour using a fixed protocol: time-and-motion studies, mystery shopping, safety walks.
  • Interviews - from standardised to open formats such as the depth interview; the more open the format, the more the interviewer becomes part of the instrument.
  • Document and record analysis - invoices, ERP logs, call recordings, annual reports; text can be coded with methods like qualitative content analysis.

The choice among them is part of research design and of data collection methods in general. In quantitative research the question "what is the measurement method?" is answered by the operational definition of a construct: what exactly gets counted, rated or timed. For organisational metrics the same thinking underlies the measurement of performance. Whether such measures capture what they claim to is a question of validity and reliability.

Selecting a measurement method

There's rarely one correct method. There is usually one that fits the purpose. Typical selection criteria:

  • Accuracy needed - trueness (closeness of the average to a reference value) and precision (closeness of replicate results). The VIM keeps these apart and warns against using "accuracy" for trueness[1].
  • Measuring range and resolution - a method perfect at 10 mg may be useless at 10 kg.
  • Tolerance to be checked - a common rule of thumb in industry is that measurement variation should be small compared with the tolerance; MSA manuals give specific acceptance thresholds.
  • Speed and cost - including sample preparation, operator time and consumables.
  • Destructiveness - can the item be sold after the test?
  • Traceability - can results be related to a reference "through a documented unbroken chain of calibrations" (VIM 2.41)?
  • Sensitivity to environment and operator - temperature, vibration, skill.
  • Regulatory or customer requirements - a contract or a standard often prescribes the method outright.

A small point that's easy to forget: the method itself carries uncertainty. The GUM calls the component associated with the method "perhaps the most difficult uncertainty component to evaluate", because other methods, possibly unknown, might give systematically different results of apparently equal validity. Its advice is practical: determining the same measurand by different methods, or by the same method in different laboratories, often provides valuable information about that uncertainty[5].

Method validation and comparison

In the VIM, validation is "verification, where the specified requirements are adequate for an intended use" (definition 2.45)[1]. Verification asks "does it meet the specification?"; validation asks "is the specification right for this job?"

ISO/IEC 17025:2017, the standard for the competence of testing and calibration laboratories, makes this operational. Under its clause on selection, verification and validation of methods, a laboratory verifies that it can properly perform a standard method before using it, and validates non-standard methods, laboratory-developed methods, and standard methods used outside their intended scope or otherwise modified[6]. Validation studies typically examine characteristics such as working range, selectivity, trueness, precision and detection limits, to the extent the intended use requires.

The ISO 5725 series, Accuracy (trueness and precision) of measurement methods and results, supplies the statistical backbone. Part 1 sets out the general principles, definitions and statistical model; other parts give the calculations for interlaboratory studies[7]. Precision is examined under defined conditions. Repeatability conditions mean the same procedure, operators, measuring system, location and a short period of time; reproducibility conditions mean different locations, operators and measuring systems (VIM 2.20 and 2.24)[1]. Gauge R&R studies in measurement system analysis use the same split. The NIST/SEMATECH e-Handbook covers check standards, gauge studies and bias in its chapter on measurement process characterization[8]. Detailed treatment of random and systematic error and of attenuation effects is in the article on measurement error.

Comparing two methods: Bland–Altman analysis

What if a company wants to replace a slow reference method with a faster one? The standard tool comes from medicine. In 1986 J. Martin Bland and Douglas G. Altman published in The Lancet a paper on assessing agreement between two methods of clinical measurement[9]. Their approach is simple:

  1. Measure the same items with both methods.
  2. For each item compute the difference between methods and the mean of the two readings.
  3. Plot differences against means (the Bland–Altman plot) and look for patterns, such as differences growing with size.
  4. Compute the mean difference d̄ (the bias) and the standard deviation of differences s.
  5. Report the 95% limits of agreement:

\[ LoA = \bar{d} \pm 1.96 \cdot s \]

If the differences are roughly normally distributed, about 95% of them should fall within these limits. Whether the limits are acceptable isn't a statistical question at all. It's a practical one, decided against the tolerance or the decision the measurement supports.

Example of comparing measurement methods

The following data are hypothetical and serve only to illustrate the calculation.

A producer of wood pellets checks moisture content by oven drying, a destructive reference method that takes hours. The quality manager wants to know if a hand-held capacitance meter, which gives a reading in seconds, could replace it for routine checks. Eight samples are split and measured by both methods (moisture in %):

Sample Oven drying A (%) Meter B (%) Difference B − A Mean (A+B)/2
1 8.2 8.6 0.4 8.40
2 9.5 9.7 0.2 9.60
3 10.1 10.8 0.7 10.45
4 7.6 7.9 0.3 7.75
5 11.3 11.6 0.3 11.45
6 9.0 9.6 0.6 9.30
7 8.8 8.9 0.1 8.85
8 10.6 11.2 0.6 10.90

Calculations:

  • Sum of differences: 0.4 + 0.2 + 0.7 + 0.3 + 0.3 + 0.6 + 0.1 + 0.6 = 3.2
  • Mean difference (bias): 3.2 / 8 = 0.40 percentage points. The meter reads high.
  • Deviations from the mean difference: 0.0, −0.2, +0.3, −0.1, −0.1, +0.2, −0.3, +0.2. Their squares add up to 0 + 0.04 + 0.09 + 0.01 + 0.01 + 0.04 + 0.09 + 0.04 = 0.32.
  • Standard deviation of differences: s = √(0.32 / 7) = √0.0457 ≈ 0.214.
  • Limits of agreement: 0.40 ± 1.96 × 0.214 = 0.40 ± 0.42, i.e. from about −0.02 to +0.82 percentage points.

What does the manager learn? The meter reads about 0.4 points high, a bias that can be corrected by subtracting 0.4 from each reading. After correction, single readings may still differ from oven drying by roughly ±0.42 points. With a 10% moisture limit and a plant running at 9.8%, that's not good enough. At 8%, it probably is. And eight samples are very few; a real study would use many more across the whole working range.

For comparison, the Pearson correlation between columns A and B is about 0.99. Impressive. And largely irrelevant, as the next section explains.

Advantages of standardised measurement methods

Using an agreed, documented method instead of everyone's personal approach brings practical benefits:

  • Results from different shifts, plants, suppliers and laboratories become comparable.
  • Disputes between buyer and seller can be settled by reference to the method named in the contract.
  • Precision data from interlaboratory studies (ISO 5725 style) tell users what scatter to expect before they start.
  • Training is faster and operator influence smaller.
  • Trend data stay meaningful over years, which matters for control charts and process capability studies.
  • In Six Sigma, statistical process control and everyday quality control, a stable, known method is a precondition for trusting process data at all. It's standardization applied to measuring.

Limitations and common mistakes

Methods fail in predictable ways. The most frequent:

  • Using correlation to compare methods. Bland and Altman argued that the correlation coefficient measures the strength of a relation between two variables, not the agreement between them. A change of scale doesn't affect correlation, however big the change; and correlation depends on the range of the true quantity in the sample, so a wide spread of items produces high r almost automatically[9]. The pellet example shows it: r ≈ 0.99 alongside a systematic bias of 0.4 points.
  • Confusing method with procedure. "We use the ISO method" says little if the local procedure skips the conditioning step.
  • Validating once, then forgetting. New raw materials, new operators or a software update can move a method outside the conditions it was validated for.
  • Ignoring method uncertainty. Repeating a method many times reduces random scatter but tells nothing about bias built into the method itself.
  • Treating ordinal data as interval data without thinking. Averaging five-point satisfaction ratings is common practice; whether it's justified depends on the scale and the decision at stake.
  • No traceability. Without a calibration chain, a method may be precise yet anchored to nothing.
  • Overlooking reliability in soft measures. Questionnaires and observation protocols need their own checks, described under reliability of measurement.

References

Footnotes

  1. ↑ JCGM 200:2012, International vocabulary of metrology (VIM), 3rd edition, BIPM, 2012, definitions 2.1, 2.4, 2.5, 2.6, 2.13-2.15, 2.20, 2.24, 2.41, 2.45, 3.2
  2. ↑ JCGM 100:2008, Guide to the expression of uncertainty in measurement, BIPM, 2008, Annex B, definitions B.2.7 and B.2.8 (quoting VIM:1993)
  3. ↑ JCGM 100:2008, Guide to the expression of uncertainty in measurement, BIPM, 2008, clause 4.1.1
  4. ↑ Stevens S.S., On the Theory of Scales of Measurement, Science, 1946, Vol. 103, No. 2684, pp. 677-680
  5. ↑ JCGM 100:2008, Guide to the expression of uncertainty in measurement, BIPM, 2008, clause F.2.5.1 and Note
  6. ↑ ISO/IEC 17025:2017, General requirements for the competence of testing and calibration laboratories, ISO, 2017, clause 7.2
  7. ↑ ISO 5725-1:2023, Accuracy (trueness and precision) of measurement methods and results - Part 1: General principles and definitions, ISO, 2023
  8. ↑ NIST/SEMATECH, e-Handbook of Statistical Methods, Chapter 2: Measurement Process Characterization, National Institute of Standards and Technology
  9. ↑ Bland J.M., Altman D.G., Statistical methods for assessing agreement between two methods of clinical measurement, The Lancet, 1986, Vol. 327, No. 8476, pp. 307-310
Measurement method — recommended articles
Measurement error — Measurement system analysis — Reliability of measurement — Validity and reliability — Data collection methods — Statistical process control — Quality control
Retrieved from CEOpedia