Estimating Measurement Uncertainty

For the uncertainty to be truly meaningful, it must address the entire measuring process, which may have uncertainties associated with factors such as equipment calibration, equipment resolution, operator skill, sample variation, and environmental factors. In many cases, sample variability and operator skill are the largest sources of uncertainty and they are often the only source considered when only the repeatability of a measurement is evaluated. However, a more thorough analysis will consider other sources of uncertainty. As a minimum, in addition to the repeatability evaluation, an uncertainty analysis should consider the instrumentation/standard calibration and the resolution of the instrumentation.

The following a summary of the main steps in an uncertainty evaluation.

  1. Define the measurand. (Determine exactly what needs to be measured.)
  2. Carry out the needed measurements.
  3. Evaluate the uncertainty due to the calibration standard and/or instrumentation used for the measurement.
  4. Evaluate resolution/readability of all instrumentation.
  5. Evaluate the repeatability.
  6. Identify and evaluate other sources of uncertainty.
  7. Prepare uncertainty budget documentation.
  8. Evaluate reasonability of budget.
  9. Determine combined standard uncertainty.
  10. Express the uncertainty in terms of uncertainty interval and the confidence level.

Each of these steps will be looked at in more detail in the following sections

Define the Measurand and Carry Out the Needed Measurements

The first step controlling and characterizing uncertainty in a measurement is to define the measurand as explicitly as possible. It is very important to know exactly what requires measuring. If possible, determine the required level of precision and accuracy and whether they can be obtained with the available equipment. Verify that equipment is properly calibrated and/or perform user calibration check using NIST or ISO traceable reference standard. If systematic error (bias) is found to exist, record the value so that it can be eliminated with a correction to the measurement data. It is also important to ensure that test samples are identified to prevent mix-ups and to mark measurement locations to allow for repeat measurements of the same locations. Finally, make the measurements with care and correct for any bias that has been identified.

Evaluate the Uncertainty Due to the Calibration Standard and/or Instrumentation

The next step is to review the calibration data from the calibration certificate for the calibration standard or test instrumentation that will be used for the measurement. Typically, when uncertainty is stated directly on calibration certificates it will be the expanded uncertainty with a coverage factor of two and a confidence level of 95%. The uncertainty value will need to be divided by the coverage factor to get back to the standard uncertainty before the combined uncertainty can be calculated.

If a statement of tolerance or accuracy is presented on the certification certificate, this is describing the interval of values that the true value is asserted to exist. Since nothing more is known about this interval, a uniform pdf should be used to calculate the standard uncertainty associated with it. To calculate the standard uncertainty, the half interval will be divided by √3. For example, an instrument with a reported tolerance or accuracy of ±0.004mm will have a full interval of 0.008mm and a half interval of 0.004. The standard uncertainty will be 0.008mm/2√3 or 0.004mm/√3, which is 0.0023mm.

Evaluate resolution/readability of all instrumentation

Digital instrumentation provides a discrete value but due to rounding, the true value could lie within ± 0.5 times the resolution of the display. To calculate the standard uncertainty for digital device, simply divide the display resolution by √3. See the information of uniform probability density functions for more information.>

The resolution or readability of an analog device depends on the ability to estimate to the nearest scale division mark or fraction of a division. Since there is a higher probability that the true value of the measurement will occur near the best estimate of the value than near the limits of the interval of possible values, a triangular probability density function is used. To calculate the standard uncertainty associated with a triangular pdf, the interval of possible values is divided by 2√6. As an example, consider a measurement made with a dial caliper that has division marks in 0.01mm increments. Say that it is allowable to estimate to one-half of a increment or 0.05mm. The range possible values associated with this resolution is 0.05mm or ± 0.025mm. The standard uncertainty is then 0.05mm divided 2√6 or 0.011mm. See the information of triangular probability density functions for more information

Evaluate the Repeatability

To evaluate sources of uncertainty due to factors such as sample variability, placement of the measurement instrument, and operator skill and consistency in making the measurement, a repeatability study should be conducted. A repeatability study is only useful when the measurement device is sensitive enough to produce scatter in the readings. Repeat readings that all produce the same value may improve overall confidence in the measurement, but they don't provide any additional information about the probability density function. Therefore, the measurement must be sensitive enough to produce scatter in the reading so that the shape of the distribution can be determined and the standard deviation can be properly calculated. When repeat readings produce scatter that is distributed approximately equally about the mean value of the sample, then a Gaussian pdf can be used to evaluate the repeatability of the measurement. See the Gaussian probability density functions for more information.

Identify and Evaluate Other Sources of Uncertainty

Other source of uncertainty will likely be small compared to the instrumentation and repeatability evaluations discussed above, but all identifiable sources of uncertainty should be addressed. Other possible sources of uncertainty may include: parallax, thermal expansion and other temperature effects, voltage drift and etc.

Parallax is the apparent displacement or shift in an object caused by a change in position of the observer. It is the source of error in a measurement when a scale is read at a slight angle.

Combined Standard Uncertainty

Once the standard uncertainties for all the sources of uncertainty in a measurement or set of measurements have been calculated, then the combined standard uncertainty can be calculated. The combined standard uncertainty is the total uncertainty in the measurement and can come from a combination of type A and B evaluations.

 

Express the Combined Standard Uncertainty in Terms of Uncertainty Interval
and the Confidence Level

The results of the measurement and uncertainty analysis should be reported in terms of the uncertainty interval and the confidence level. Since more than one type of pdf likely contributed to the combined uncertainty, the type of pdf used cannot be stated. Therefore, a measurement might be reported as 18.2 ± 0.15mm with a confidence level of 68%. The measurement could also be reported as 18.2 with a relative uncertainty of ± 0.0083. The uncertainty could also be expressed in terms of percent uncertainty as ± 0.83%.

Combined Standard Uncertainty
and Propagation of Uncertainty

For the uncertainty to be truly meaningful, it must address the entire measuring process, which may have uncertainties associated with factors such as equipment calibration, operator skill, sample variation, and environmental factors. When a measurement has more than one identifiable source of measurement uncertainty, then the combined standard uncertainty (uc) must be calculated.

Calculating the combined standard uncertainty is a two step process. The first step is to determine the uncertainties measured directly and the second step is combine the uncertainties using summation in quadrature, which is also known as root sum of the squares. For example, is a measurement of a measurand x, has three sources of uncertainty for which the three standard uncertainties u1(x) , u2(x) and u3(x) have been determined, then the combined standard uncertainty uc(x) for the measurement is given by:

http://www.ndt-ed.org/GeneralResources/Uncertainty/Graphics/Combined_Uncert2.jpg

Uncertainty contributions from both Type A and Type B evaluations may be combined as long as they are expressed in similar terms before they are combined. Thus, all the uncertainties must be expressed as one standard uncertainty and in the same units.

Propagation of Uncertainty

Many physical quantities are not determined from a single direct measurement but instead are calculated by combining two or more separate measurements. Therefore, it is important to understand how measurement uncertainty propagates when mathematical operations are performed on measured quantities, so that a final combined uncertainty can be calculated. Consider the determination of the velocity of a sound wave as it travels through a medium. The velocity (V) is calculated by dividing the measured distance (d) traveled by the measured time (t) that it took the sound to travel the distance. This calculation of velocity is easy enough but the measured quantities (d and t) each have a measurement uncertainty that must be combined to arrive at an uncertainty for the velocity calculation.

The propagation of uncertainty is treated differently depending on the mathematical operation(s) performed. The simplest case is where the result is the sum of a series of measured values (either added together or subtracted).  The combined standard uncertainty is found by squaring the uncertainties, adding them all together, and then taking the square root of the total. For more complicated cases, such as multiplication and division where mixed units are often involved, it is necessary to work in terms of relative uncertainties. The formula for calculating the combined standard uncertainty for basic mathematical operations are shown in the table below.

Mathematical Operation Performed

Equation Form

  Formula for Calculation of Combined Standard   Uncertainty

Addition or Subtraction

  Z± u(z) = (X ± u(x)) + (Y ± u(y)) or
  Z± u(z) = (X ± u(x)) - (Y ± u(y))

 http://www.ndt-ed.org/GeneralResources/Uncertainty/Graphics/Combined_Uncert2b.jpg

Multiplication or Division

  Z ± u(z) = (X ± u(x)) x (Y ± u(y)) or
  Z ± u(z) = (X ± u(x)) / (Y ± u(y))

 http://www.ndt-ed.org/GeneralResources/Uncertainty/Graphics/Combined_Uncert3A.jpg
 See Note 1

Power (Squared)

  Xn± u(xn)                 X2 ± u(x2)

http://www.ndt-ed.org/GeneralResources/Uncertainty/Graphics/Combined_Uncert7.jpghttp://www.ndt-ed.org/GeneralResources/Uncertainty/Graphics/Combined_Uncert4.jpg
 See Note 1

Radical
(Square Root)

http://www.ndt-ed.org/GeneralResources/Uncertainty/Graphics/Combined_Uncert9.jpg     http://www.ndt-ed.org/GeneralResources/Uncertainty/Graphics/squareroot.jpg

  http://www.ndt-ed.org/GeneralResources/Uncertainty/Graphics/Combined_Uncert8.jpg        http://www.ndt-ed.org/GeneralResources/Uncertainty/Graphics/Combined_Uncert5.jpg
 See Note 1

Mixed (Addition, Division, Square, and Square Root)

http://www.ndt-ed.org/GeneralResources/Uncertainty/Graphics/Mixed1.jpg

http://www.ndt-ed.org/GeneralResources/Uncertainty/Graphics/Mixed2.jpg
 See Note 1

Note 1: The result of this calculation is the relative combined uncertainty. The absolute combined uncertainty can be calculated by multiplying uc by the best approximation of the measurand.

Correlation

The equations in the table above or only valid if the contributing uncertainties are not correlated. Factors leading to measurement error are often independent, but sometimes they are correlated of inter-related. For example, a temperature shift could have a similar effect on several uncertainty contributors. If two or more sources of uncertainty are believed to be correlated, consult the references for additional information on dealing with the correlation.

The Uncertainty Budget

An uncertainty budget is simply a way of organizing and summarizing the uncertainty analysis in tabular form. An uncertainty budget lists all the contributing components of uncertainty and these components are used to calculate the combined standard uncertainty for the measurement. The table can consist of as few as two columns, one for listing the source of uncertainty and the second for recording the standard uncertainty. However, more involved tables such as the one shown below can be helpful.

Source of Uncertainty

  Value ±

  Shape of pdf

  Divisor

Standard Uncertainty

Calibration uncertainty (@k=2)

  1.50 mm

  Normal

          2
(to reduce k=2 to 1)

  0.75 mm

Resolution (size of divisions)

  0.5 mm

  Uniform

        √3

  0.29 mm

Standard uncertainty of mean
(10 repeated readings)

  0.38 mm

  Normal

           1

  0.39 mm

 

 

 

 

 

Combined standard uncertainty

 

  Assumed normal

 

  0.90 mm

Expanded uncertainty (k=2)

 

  Assumed normal  

 

  1.80 mm


Example Determination of Combined Uncertainty
for Simple Subtraction Calculation

Consider the need to determination of the remaining wall thickness between the bottom of a drilled hole and the surface. This determination would require the depth of the hole to be measured and subtracted from the measured value of the total thickness of the block. Five readings for each measurement were taken and summarized in the table below.

Reading #

Specimen Thickness, mm

Hole Depth, mm

1

21.06

16.60

2

21.02

16.68

3

21.10

16.58

4

21.04

16.56

5

21.08

16.64

Mean

21.06

16.61

For this example, two possible source of uncertainty in the measurement will be considered: the resolution of the dial gage and the repeatability of the measurement. Since there are multiple sources of uncertainty in this calculation, the evaluation process will be broken down into the following steps:

  1. Calculate the standard uncertainty due to the resolution of the dial gage.
  2. Calculate the standard uncertainty due to the repeatability of each measurement.
  3. Calculate the combined standard uncertainty for each of the two measurements.
  4. Calculated the combined standard uncertainty for the calculated remaining wall thickness.
  5. State the uncertainty in terms of an uncertainty interval, coverage factor and level of confidence.

Uncertainty of Individual Measurements Due to Resolution of Dial Gage
First, consider the uncertainty of each of the two measurements separately. For this example, two possible source of uncertainty in the measurement will be considered: the resolution of the dial gage and the repeatability of the measurement. The dial has a resolution of 0.02mm and since this is an analog device, a triangular pdf will be used to determine the standard uncertainty due to the device resolution. Since the dial is being read to the nearest division, a reading could be off by ± 0.01mm. Since 0.01mm is half of the interval of possible values that would be rounded up or down to get to a division marking, the standard uncertainty due to the resolution of the caliper will by 0.01/√6 or ±0.00408mm. This will be the same for both the specimen thickness and the hole depth measurements and since it is an intermediate result, it will be left unrounded.

Uncertainty of Individual Measurements Due to Measurement Repeatability
The uncertainty due to the measurement repeatability is the standard deviation of the mean of the repeat readings. The standard deviation of the specimen thickness measurement is ±0.031623mm. This value is divided by the square root of the number of measurements to produce a standard uncertainty of ±0.014142mm. The standard deviation of the hole depth measurement is ±0.048166mm and the standard uncertainty is ±0.021541mm. Again, since these standard uncertainties are intermediate results, they were left unrounded.

Combined Uncertainty of Individual Measurements
The combined standard uncertainty for the specimen thickness measurement is then the root sum of the squares of 0.00408mm and 0.014142mm and this equals ±0.01472mm. The combined standard uncertainty for the hole depth measurement is then the root sum of the squares of 0.00408mm and 0.021541mm, which is ±0.02192mm.

Combined Uncertainty of Calculated Remaining Wall Thickness
The remaining wall thickness is the specimen thickness minus the hole depth, which is 21.06mm minus 16.61mm, equals 4.45mm. The combined standard uncertainty for this value is then root sum of the squares of 0.01472mm and 0.02192mm or ±0.0264mm. To increase the confidence level to 95%, 0.0264 would be multiplied by a coverage factor of two to get 0.0528. Since this is the final combined uncertainty, this value will be rounded to 0.053mm.

State the Uncertainty in Terms of an Uncertainty Interval and Level of Confidence.
The final value for the remaining wall thickness would then be reported as 4.45mm ± 0.053mm with a 95% confidence level.