Issue 
Int. J. Metrol. Qual. Eng.
Volume 8, 2017



Article Number  14  
Number of page(s)  9  
DOI  https://doi.org/10.1051/ijmqe/2017014  
Published online  24 May 2017 
Research Article
Comparison of GUM and Monte Carlo methods for the uncertainty estimation in hardness measurements
National Institute of Standards,
Tersa St, ElHaram, Box 136 Code 12211,
Giza, Egypt
^{⁎} Corresponding author: goudamohamed15@yahoo.com
Received:
7
January
2017
Accepted:
2
May
2017
Monte Carlo Simulation (MCS) and Expression of Uncertainty in Measurement (GUM) are the most common approaches for uncertainty estimation. In this work MCS and GUM were used to estimate the uncertainty of hardness measurements. It was observed that the resultant uncertainties obtained with the GUM and MCS without correlated inputs for Brinell hardness (HB) were ±0.69 HB, ±0.67 HB and for Vickers hardness (HV) were ±6.7 HV, ±6.5 HV, respectively. The estimated uncertainties with correlated inputs by GUM and MCS were ±0.6 HB, ±0.59 HB and ±6 HV, ±5.8 HV, respectively. GUM overestimate a little bit the MCS estimated uncertainty. This difference is due to the approximation used by the GUM in estimating the uncertainty of the calibration curve obtained by least squares regression. Also the correlations between inputs have significant effects on the estimated uncertainties. Thus the correlation between inputs decreases the contribution of these inputs in the budget uncertainty and hence decreases the resultant uncertainty by about 10%. It was observed that MCS has features to avoid the limitations of GUM. The result analysis showed that MCS has advantages over the traditional method (GUM) in the uncertainty estimation, especially that of complex systems of measurement. MCS is relatively simple to be implemented.
Key words: uncertainty / normal distribution / Monte Carlo / guideline for uncertainty of measurement / correlation
© G.M. Mahmoud and R.S. Hegazy, published by EDP Sciences, 2017
This is an Open Access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/4.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.
1 Introduction
The guide to the expression of measurement uncertainty (GUM, JCGM 100) and the propagation of distributions by a Monte Carlo method (GUMS1, JCGM 101) are two of the most widely used documents concerning measurement uncertainty evaluation in metrology. Both documents describe three phases: (a) the construction of a measurement model, (b) the assignment of probability distributions to quantities, and (c) a computational phase that specifies the distribution for the quantity of interest. The two approaches described in these two documents agree in the first two phases but employ different computational approaches, with the GUM using linearization to simplify the calculations [1]. The guide to the Expression of Uncertainty in Measurement (GUM) requires, “that the results of a measurement have been corrected for all recognized significant systematic effects and that every effort has been made to identify such effects”, before the issue of evaluating their uncertainty is tackled [2]. GUM provides a framework and procedure for evaluating and expressing measurement uncertainty. GUM procedure has two main limitations. Firstly, the way that the coverage interval is constructed to contain values of the measurand with a stipulated coverage probability is approximate. Secondly, insufficient guidance is given for the multivariate case in which there is more than one measurand [3]. Figure 1 shows the schematic to explain the main idea of GUM which mainly depends on the uncertainty propagations [4].
Monte Carlo Simulation (MCS) is the second method to estimate the uncertainty of measurements, where evaluating the measurement uncertainty by the MCS method can be carried out by establishments of the model equation for the measurand in function of the individual parameters of influence, then selecting the significant sources of uncertainty, identification of the probability density functions corresponding to each source of uncertainty selected, and selecting the number Monte Carlo trials, and finally calculating the M results by applying the equation that was defined for the measurand [5].
MCS process is illustrated in Figure 2a and b in comparison with the propagation of uncertainties used by the GUM.
Figure 2a shows an illustration representing the propagation of uncertainties. In this case, three input quantities are presented x_{1}, x_{2} and x_{3} along with their respective uncertainties u(x_{1}), u(x_{2}) and u(x_{3}). As can be noted, only the main moments (expectation and standard deviation) of the input quantities are used in the propagation. In Figure 2b, while propagating density (PDFs), no approximations are made and the whole information contained on the input distributions is propagated to the output [5].
The aims of this present research article are to investigate the differences between GUM and MCS as methods of uncertainty estimation of hardness and to detect the effects of inputs correlation on the uncertainty budget.
Fig. 1 GUM and propagation of uncertainties. 
Fig. 2 Schematic to the main idea of MCS. 
2 Theoretical principle
2.1 Uncertainty classifications
To investigate the difference between GUM and MCS, the resultant budget uncertainties of Brinnell and Vickers hardness (HV) measurements were used as an application. For statistical analysis normal PDFs were proposed. The uncertainty estimated from normal distribution is believed to lie in the interval defined by U with a level of confidence of approximately 95%. For normal distribution, μ is the expectation or mean of the distribution, and the shaded areas represent ±1.96 standard uncertainty u_{stnd} about the mean, and ±u_{stnd}, encompasses about 95% of the distribution (see Fig. 3).
Each input source of the uncertainty should be classified as type A or type B where type A evaluation of standard uncertainty is based on any valid statistical method for treating data and can be calculated by the following equations: (1) (2) where X_{i} is an input quantity whose value is estimated from n independent observations X_{i,k} of X_{i} obtained under the same conditions of measurement, and the standard uncertainty u(x_{i}) to be associated with x_{i} is the estimated standard deviation of the mean. For type B standard uncertainty, the evaluation of standard uncertainty is usually based on scientific judgment using all of the relevant information available, that is, the uncertainty is either obtained from an outside source, or obtained from an assumed distribution.
Fig. 3 The normal distribution, and standard deviation with the confidence level. 
2.2 Uncertainty in measurements with and without correlation
If the functional relationship between the measurand Y and the input quantities X in a measurement process is given by [6]; (3) The function f is used to calculate the output estimates: y, of the measurand; Y, using the estimates of X_{1}, X_{2}, X_{3}, … , X_{N} for the values of the N input quantities. (4)The conventional estimation can be illustrated using a simple equation with y as a continuous function of x_{1}, x_{2}. y is approximated using a polynomial approximation or a 2nd order Taylor's series expansion about the means: (5)where are mean values and Z is the remainder: (6)As the partial derivatives are computed at the mean values they are the same for all i = 1, 2, …, N. All the higher terms are normally neglected at Z = 0, so equation (5) becomes; (7)If f(x_{1}, x_{2}) is a linear function, then the second order partial derivatives in equation (6), are zero, so Z = 0. Both linearity and small uncertainty are prerequisites of conventional method of uncertainty estimation described below. The standard deviations σ(X_{1}), σ(X_{2}) are referred by GUM [7], as the standard uncertainties associated with the input estimates X_{1}, X_{2}. The standard uncertainty in y can be obtained by Taylor equation [8]; (8)This equation gives the uncertainty as a standard deviation irrespective of whether or not the measurements of x_{1}, x_{2} are independent and of the nature of the probability distribution. In the case of correlated inputs then equation (3) can be written in terms of the correlation coefficient ; (9)
The correlation coefficient can be calculated by the following equation [9]: (10) As a direct consequence of its definition, the correlation coefficient r is restricted to the range ‒1 through zero to +1. When r = 0, the correlation is zero, when r = +1, the correlation is perfect and positive, and when r = −1 the correlation is perfect and negative.
The partial derivatives are called sensitivity coefficients, which give the effects of each input quantity on the final results (or the sensitivity of the output quantity to each input quantity). The term, expanded uncertainty is used in GUM to express the % confidence interval about the measurement result within which the true value of the measurand is believed to lie and is given by: (11) where k is the coverage factor on the basis of the confidence required for the interval, (12)If it was considered that y = x_{1} + x_{2} is a linear operation it was reported that there is compatibility between MCS and GUM and it was demonstrated that at correlated or uncorrelated inputs MCS gives identical results as given in equations (5) and (9). The results of y are linear functions in terms of x_{1} and x_{2}. As the first order partial derivatives are all equal to ±1, the square is equal to unity. The second order partial derivatives are both equal to zero.
To quantify the distribution of the results of Skewness and Kurtosis, where Skewness quantifies how symmetrical the distribution, it can be calculated from the following equation: (13) where

Positive Skewness indicates a long right tail;

Negative Skewness indicates a long left tail;

Zero Skewness indicates a symmetry around the mean.
Kurtosis quantifies whether the shape of the data distribution matches the Gaussian distribution and it can be calculated from the following equation: (14)

Positive excess Kurtosis indicate flatness (long, fat tails);

Negative excess Kurtosis indicates peakedness.
2.3 Brinell hardness (HB) results
This test is executed by applying a load on a sphere made of a hard material over the surface of the test sample (Fig. 4) [10].
During the test the sphere will penetrate through the sample leaving an indented mark upon unloading. The diameter of this indentation is inversely proportional to the hardness of the material of the sample. The model used for the HB is represented in equation (15) [11]. (15) where F is the applied load (N), D is the indenter diameter (mm) and d is the diameter of the indentation mark (mm).
Fig. 4 Schematic represents the Brinell hardness test. 
2.4 HV results
HV is a measure of the hardness of a material, calculated from the size of an impression produced under load by a pyramidshaped diamond indenter (Fig. 5) [12].
During the test the square pyramid indenter will penetrate through the sample leaving an indented mark upon unloading. The two diagonals of this indentation are measured and the mean of them was calculated.
The model used here for the HV is represented in equation (16) [12]. (16) where F is the applied load (kgf), and d is the mean of d_{1} and d_{2} of the indentation mark (mm).
Fig. 5 Schematic represents Vickers hardness test. 
3 Application results
3.1 Procedure to estimate the uncertainty of HB
To estimate the uncertainty it is required to detect all the input sources of uncertainty of HB measurements. These sources can be summarized as shown in Figure 6.
After detecting these parameters, measurements were performed and the results analysis was conducted taking into account the calculation of these parameters as normal distribution, which is can be summarized in Table 1.
Fig. 6 The cause–effect diagram for HB. 
Results analysis of measurements for HB.
3.1.1 GUM procedure to estimate HB uncertainty with and without correlated inputs
To estimate the budget uncertainty in accordance with GUM, sensitivity coefficient should be calculated in accordance with the following equations: (17) (18) (19) The uncertainty budget of HB is summarized in Tables 2 and 3.
Uncertainty budget for HB by GUM without correlated inputs.
Uncertainty budget for HB by GUM with correlated inputs.
3.1.2 MCS procedure to estimate HB uncertainty and without correlated inputs
10 000 iterations were performed on each parameter contributing the uncertainty budge using equation (13) to estimate the budget uncertainty (see Tab. 4; Fig. 7).
Iterations of the input parameters for uncertainty budget of HB by MCS.
Fig. 7 Results of HB by MCS. 
3.1.3 Summary of the results of uncertainty budget estimation of HB by MCS
MCS for HB without correlated inputs
Mean value: 100.1 HB.
Expanded uncertainty: 0.67 HB.
MCS for HB with correlated inputs
Mean value: 100.1 HB.
Expanded uncertainty: 0.59 HB.
3.2 Procedure to estimate the uncertainty of HV
The detected input sources of uncertainty of HV measurements can be summarizes as shown in Figure 8 and Table 5.
Fig. 8 The cause–effect diagram for HV. 
Results analysis of Vickers hardness measurements.
3.2.1 GUM procedure to estimate HV uncertainty with and without correlated inputs
To estimate the uncertainty budget of HV sensitivity coefficient should be identified by the following equations (see Tabs. 6 and 7). (20) (21)
Uncertainty budget for HV by GUM without correlated inputs.
Uncertainty budget for HV by GUM with correlated inputs.
3.2.2 MCS procedure to estimate HV uncertainty with and without correlated inputs
10 000 iterations were performed on each parameter contributing the uncertainty budge using equation (16), to estimate the budget uncertainty (see Tab. 8; Fig. 9).
Iterations of the input parameters for uncertainty budget of HV by MCS.
Fig. 9 Result of the Monte Carlo simulation for HV. 
3.2.3 Summary of the results of uncertainty budget estimation by MCS
MCS for HV without correlated inputs
Mean value: 401.9 HB.
Expanded uncertainty: ±6.5 HV.
MCS for HV with correlated inputs
Mean value: 401.9 HB.
Expanded uncertainty: ±5.8 HV.
4 Discussion
From the previous figures and tables it was noted that the resultant expanded uncertainties (at 95% confidence levels) obtained with the GUM and MCS without correlated inputs for HB were ±0.69 HB, ±0.67 HB and for HV were ±6.7 HV, ±6.5 HV, respectively. The estimated expanded uncertainties with correlated inputs by GUM and MCS were ±0.6 HB, ±0.59 HB and for HV were ±6 HV, ±5.8 HV, respectively. GUM Framework overestimates a little bit the MCS estimated uncertainty. The main cause of this difference is the approximation used by the GUM Framework in estimating the budget uncertainty of the calibration curve produced by least squares regression.
The correlations between inputs have significant effects on the estimated uncertainties. At GUM procedure the estimated uncertainty of HB without correlated inputs was 0.69 HB and 0.6 HB at correlated inputs. For HV the estimated uncertainty without correlated inputs was 6.7 HV and 6 HV at correlated inputs. The same results were obtained in the case of MCS investigation.
Skewness calculation showed small value and hence a symmetrical distribution for the obtained results.
The shape of Kurtosis quantifications showed that the data distribution matches the Gaussian distribution.
Also, it was observed that MCS has features to avoid the limitations and assumptions of the GUM framework. The resulting analysis shows that MCS has many advantages over conventional method (GUM) in uncertainty estimation, especially that of complex measurement systems. MCS is relatively simple to implement; there is no need for complex mathematics related to calculating sensitivity coefficient by partial differentiation and also it was demonstrated that the MCS is relatively compatible with the conventional uncertainty estimation methods of linear systems and systems that have small uncertainties.
5 Conclusions
From this research article it was concluded that:

the expanded uncertainty results estimated with the GUM Framework and the MCS showed no significant differences. In all the cases the estimated uncertainty using the GUM approach slightly overestimated the results obtained with the MCS;

the correlations between inputs have significant effects on the estimated uncertainties. Thus the correlation between inputs decreases the contribution of these inputs in the budget uncertainty and hence decreases the resultant uncertainty by about 10%, and this value is depending on the correlation coefficient value;

the result analysis shows that the MCS has numerous advantages over the traditional method (GUM) in the estimation of uncertainty, especially that of complex systems of measurements. There is no need for complex mathematics related to calculating sensitivity coefficient by partial differentiation;

it was demonstrated that the MCS is relatively compatible with the GUM as a conventional uncertainty estimation methods of linear systems and systems that have small uncertainties.
References
 A.B. Forbes, Approaches to evaluating measurement uncertainty, Int. J. Metrol. Qual. Eng. 3, 71–77 (2012) [CrossRef] [EDP Sciences] (In the text)
 F. Pavese, Corrections and input quantities in measurement models, Int. J. Metrol. Qual. Eng. 3, 155–159 (2012) [CrossRef] [EDP Sciences] (In the text)
 P.M. Harris, M.G. Cox, On a Monte Carlo method for measurement uncertainty evaluation and its implementation, Metrologia 51, S176–S182 (2014) [CrossRef] (In the text)
 JCGM 101:2008, Evaluation of Measurement Data – Supplement 1 to the “Guide to the Expression of Uncertainty in Measurement” – Propagation of Distributions Using a Monte Carlo Method (Joint Committee for Guides in Metrology, 2008) (In the text)
 ISO/Guide, Guide to the Expression of Uncertainty in Measurement (GUM) – Supplement 1: Numerical Methods for the Propagation of Distributions (1998) (In the text)
 M. Basil, C. Papadopoulos, D. Sutherland, H. Yeung, Application of probabilistic uncertainty methods (Monte Carlo Simulation) in flow measurement uncertainty estimation, in Flow Measurement International Conference (2001) (In the text)
 ISO/GUM, Guide to the Expression of Uncertainty in Measurement – ISO (1995), ISBN 9267101889 (In the text)
 J.R. Taylor, An Introduction to Error Analysis/The Study of Uncertainties in Physical Measurements, 2nd edn. (University Science Books, Sausalito, CA, 1997), ISBN 0935702423 (In the text)
 I. Farrance, R. Frenkel, Uncertainty of measurement: a review of the rules for calculating uncertainty components through functional relationships, Clin. Biochem. Rev. 33, 49–75 (2012) [PubMed] (In the text)
 EUROLAB Technical Report 1/2006, Guide to the Evaluation of Measurement Uncertainty for Quantitative Test Results (2006) (In the text)
 ISO 65062:2014, Metallic Materials – Brinell Hardness Test – Part 2: Verification and Calibration of Testing Machines (In the text)
 ISO 65072:2005, Metallic Materials – Vickers Hardness Test – Part 2: Verification and Calibration of Testing Machines (In the text)
Cite this article as: Gouda M. Mahmoud, Riham S. Hegazy, Comparison of GUM and Monte Carlo methods for the uncertainty estimation in hardness measurements, Int. J. Metrol. Qual. Eng. 8, 14 (2017)
All Tables
All Figures
Fig. 1 GUM and propagation of uncertainties. 

In the text 
Fig. 2 Schematic to the main idea of MCS. 

In the text 
Fig. 3 The normal distribution, and standard deviation with the confidence level. 

In the text 
Fig. 4 Schematic represents the Brinell hardness test. 

In the text 
Fig. 5 Schematic represents Vickers hardness test. 

In the text 
Fig. 6 The cause–effect diagram for HB. 

In the text 
Fig. 7 Results of HB by MCS. 

In the text 
Fig. 8 The cause–effect diagram for HV. 

In the text 
Fig. 9 Result of the Monte Carlo simulation for HV. 

In the text 
Current usage metrics show cumulative count of Article Views (fulltext article views including HTML views, PDF and ePub downloads, according to the available data) and Abstracts Views on Vision4Press platform.
Data correspond to usage on the plateform after 2015. The current usage metrics is available 4896 hours after online publication and is updated daily on week days.
Initial download of the metrics may take a while.