Issue 
Int. J. Metrol. Qual. Eng.
Volume 7, Number 4, 2016



Article Number  405  
Number of page(s)  14  
DOI  https://doi.org/10.1051/ijmqe/2016017  
Published online  24 November 2016 
Research Article
Evaluation of indices for the measurement of quality in health systems
^{1}
Institute of Biostructure and Bioimaging, National Research Council (IBBCNR),
Rome, Italy
^{2}
Campus BioMedico University and LUISS Guido Carli,
Rome, Italy
^{3}
Campus BioMedico University,
Rome, Italy
^{4}
Department of Anaesthesia and Resuscitation, AO S.CamilloForlanini,
Rome, Italy
^{⁎} Corresponding author: luca.pontecorvi@gmail.com
Received:
1
June
2016
Accepted:
13
September
2016
In recent decades different numeric indicators and methods have been used to qualify structured index that express health service components quality performance. The first step of this work is the definition of a model of the system under examination in order to define its measurable constituent entities (dimensions). Among the common measured clinical parameters, a robust metrics, characterizing the constituent entities and the best opportunity tools for the characterization of the results, have been identified. The last step is multimodal analysis of the results. Using this approach the paper presents an application in the evolution of the cardiological intensive care unit (ICU). Particularly, the attention of this paper is focused on the evolving of the medical team.
Key words: health services / metrics / statistical quality control
© EDP Sciences, 2016
1 Introduction
In recent decades the term quality has been introduced in to the common language as an essential part of clinical management. The intents of quality approach in healthcare are mainly focused towards the identification of needs, and the evaluation of the service through measurements and indices suitable to assess the performance of the system [1,2]. As in all sectors the use of suitable quality metrics is crucial. Then must be defined tools to evaluate and combine indices to have structured methodologies that express a global evaluation of the quality performance.
Quality indicators can be classified in several different ways, in addition they can be compared and analyzed using different approaches. Some indicators can take contributions from assessed classifications, or more indicators can be used simultaneously according to the rules of data stratification [3]. Indicators can describe events and rates of service's utilization by the population [4]. Higher rates of several indicators may be interpreted as efficiency in functioning, whereas an indicator such as the death rate, which is usually referred to specific clinical situations, may serve to assess the risk of death after medical intervention. A widespread approach relies on the definition and use of benchmarks, which allows a comparison between indicators related to homogeneous systems and processes [5]. Indicators can be divided into subgroups to analyse the system, even through simplified models offer the possibility to identify the areas of the health system in a conceptual framework that link these areas to correlating indicators for an overall analysis. The goal is a first characterization of the individual components and the successive study of interrelationship between them to improve the outcome in the individual patient. For this the model proposed by Donabedian [6] and integrated by Handler et al. [7] is widely accepted and proposed in many studies [1] for the characterization and the measure of the quality in healthcare.
An accepted fact in managing metrics for quality is to put in evidence that in any process there is a background noise due to cumulative effects of many small, essentially unavoidable causes, i.e. stable system changes. Thus the estimation of useful indicators requires a phase of analysis in which statistical techniques can be deployed. Between more common approaches to evaluate systems dynamic of statistical quality control (SQC) it is useful to detect the occurrence of assignable causes of process shift. Therefore causes of improvement can be assigned and highlighted with the final aim of monitoring improvement causes/actions. Key concepts of SQC include the idea of a rational sub process analysis and the use of tools such as histograms, run charts, and control chart signals. The tools of SQC help to create an environment in which the stakeholders of each area seek continuous improvement for their activities. Elsewhere SQC techniques are applied to predicting activities [8] and to understanding whether there are changes in the care and in the management of altered, patient outcome [9] in this clinical field. In this paper an example of the application of statistical methods to health service to quantify new paradigms and procedures in health service is provided [10]. Some practical issues in the implementation of SQC applied to evaluate the ICU of a cardiac surgery department, are shown. The final aim is to provide a methodology in order to study how change in healthcare alters patient outcome [9,11].
2 Quality measurements in health systems
As stated quality measurements in healthcare need the identification of indicators that allow an objective assessment of the service under examination. This is particularly relevant for new developments in domains, including those concerning diagnostic/treatment procedures, interoperability, and the use of new medical devices [12]. Besides quality metrics it is necessary to synthesize data in a conceptual model contemplated for global quality assessment and health system monitoring [7]. The conceptual system, here considered to measure the quality of the health system, is derived from the measurement approach proposed by Donabedian and widely accepted in the literature [13]. The approach by Donabedian combines several issues, related to the organizational structure, processes, and outcomes, in a comprehensive clinical model. In particular, the health system is split into four conceptual components: mission, structural capacity, processes and outcomes, all interacting with the macro context [2,5,7,14]. From this division in different areas the evaluation of reference indicators and/or parameters, is a technique widely used in healthcare. As a mater of fact the conceptual system here adopted simplifies this approach by considering the simplest model derived from the classical process approach and performance improvement of ISO 9000 standard and highlights as indicators of quality a set of parameters related to the structure, process and outcomes [1,3] (adopted (Fig. 1)). The structure (organizational data) is characterized by the features of the hospital, operators’ skills, the characteristics of equipment and human resources, and so on. The process consists in the encounter between healthcare physicians and patients (professional quality). It starts from reservation and admission, through anamnesis, screening planning and execution, diagnosis, therapy and administration of drugs. Elsewhere [3] the indicators characterizing the process can be widely recognized clinical standards and are extracted from clinical data. The clinical outcomes (professional quality) are according to the type of clinical service. They give information on the state of health of the patient during and after treatment by the health service including the results such as: mortality, morbidity, perception and satisfaction of the patients at discharge, quality of life after treatment [15]. The evaluation can be performed on each dimension (structure, process and outcome), using different methods of measurement based on as many different indicators.
Controlling and improving quality has become an important business strategy for healthcare providers and control chart is one of the primary techniques to be adopted. This chart plots the average of measurements of a quality characteristic in samples taken from the process versus time. The charts highlight where and when the process fails in order to recognize unusual sources of variability present. When unusual sources of variability are present, sample average will plot outside the control limits. This is a signal that some investigation of the process should be made and corrective action to remove these variability must be taken [10].
Another hospital quality evaluation is the study of internal dynamics of a hospital which represent a complex nonlinear structure. The planning and managing of these resources requires a good understanding of the hospital system, therefore an advanced data analysis is proposed. Hospital Length of Stay (LoS) of inpatients has been employed as a proxy for measuring the consumption of hospital resources. Simple averages are widely used to describe LoS but parameters can be incorporated in models of resource utilization and patient flow. Thus a model that forecasts the LoS starting from the EuroSCORE (ES), an index of health status of the admitted patients, is proposed. Furthermore in order to explore the complex relationship between bed occupancy and ES values and optimize resource utilization, a model to describe the discharge frequency is used.
Fig. 1 Schematic of a model of ICU department that highlights as quality indicators a set of parameters related to the structure, process, and outcomes [1]. 
3 Data analysis
Data are from the unit of Anesthesiology and Reanimation intensive care unit of San CamilloForlanini Hospital in Rome which is part of the Department of Cardioscience. The team of the Unit was formed in 1999, and even started to systematically collect some important data about patients admitted. The data analyzed in this paper refer only to the patients coming from the Cardiac Surgery Unit. In particular, the database consists of 11770 patients (records) collected from 1999 to 2012, and the corresponding entries were: Name, Intervention Date, Age, Pathology, Sex, Urgency, Intervention, Ejection Fraction, Pulmonary Hypertension, Diabetes, Obesity, Chronic Obstructive Bronchopathy, Chronic Kidney Failure, Liver Failure, Neuropathy, Reintervention, Extracorporeal Circulation Time, Orotracheal Intubation Time, Length of Stay in Intensive Care Unit, ES, PostOperative Complications, Exitus (dead patient).
3.1 Structure
The Unit counts up to 12 beds, and the team is made up of one head physician, 19 staff physician, one head nurse, and 36 staff nurses. The activity of the Unit is aimed at supporting the whole Department (Cardiac Surgery Unit and the Vascular Surgical Unit). In addition to the wellestablished transplant activity, the Cardiac Surgery Unit ordinarily performs coronary artery bypass graft surgery (CABG), beating, replacements and repair of the aortic and mitral heart valve (Valv), surgical treatment of heart failure (AO), reconstructive surgery of the aortic root and of ascending aorta, and surgery interesting aortic arch. While, Vascular Surgical Unit ordinarily performs thoracic and abdominal surgical interventions for peripheral arterial occlusive disease, for carotid artery disease, for the lower limbs varicose veins disease, and also performs videolaparoscopic procedures, angioplasty and carotid artery stents, angioplasty and peripheral stent, and aortic endoprosthesis. As stated, the data of this paper refer only to cardiac surgery.
3.2 Process
The first process indicator considered in this paper is the number of interventions, in general and divided into different surgeries. Figure 2 shows the trend of the number of interventions in each of the above considered categories. The yearly average number of interventions is 838. If the percentage of isolated CABG interventions every year is considered, it can be observed that it becomes less and less (passing from the 48% on average of the first seven years, to the 40% of the last seven). This is a logical consequence of the fact that the Cardiac Surgery Unit is trying to treat this kind of patient not with a surgical approach, but with stenting applications, reducing the admissions to the Reanimation Unit.
The second indicator analyzed was the use of mechanical supports for cardiocirculatory function assistance divided by different types. There are various types of device, this paper considers the main one used:

Mechanical Assistance
 –
VAD (Ventricular Assist Device)
 –
BIVAD (BIVentricular Assist Device, and Artificial Heart)
 –

LEVITRONIX (Magnetic Centrifugal Pump)

ECMO (Extra Corporeal Membrane Oxygenation)

IABP (Intra Aortic Balloon Pump).
Figure 3 shows the number of devices used every year, and Figure 4 shows the trends for each type of device (note that the variable Mechanical Assistance includes VAD and BIVAD). In line with technological development both figures show an increase in the use of the mechanical support.
The third indicator is the Length of Stay (LoS): the term Length of Stay indicates the measure (generally days) of the duration of a stay in a department. The importance of LoS as an indicator in the ICU process is twofold [16]: first of all, it gives the measure of the efficiency of the process and it is used as the measure of costs of the ICU and of the global hospital (the ICU while using between 5% and 10% of the beds can consume up to 20% of hospital budgets). Secondly, the LoS can be used as an indirect indicator of the quality of care [17]. Figure 5 shows the yearly average LoS, in general and in different types of patients. The reference solid line (Y = 3.02) is the average of all admitted patients (aggregated from 1999 to 2012) and the dashed line shows the trend of average LoS for different types of patients. In Table 1, the values of the averages for each type of patients aggregated from 1999 to 2012 are reported.
Among process indicators patient risk parameters are also included that describe the clinical condition of the population admitted into the ICU. The reason for this choice is that these parameters give a description of those which are the inputs to the activities that represent processes. The risk parameters analyzed are:

age of patients;

ES of patients.
Table 2 shows the percentage of patients being part of the represented age range. The majority of patients admitted in ICU were more than 60 years old (about 70% of the whole population). Instead, in Figure 6 the percentages of patients are represented in every considered age range, divided in every year. The most important thing to observe is the increase of the percentage of patients >80 years, from an average of 3.2% in the first seven years to 6.5% in the last seven, thus showing the fact that the Unit has faced a remarkable changing of the population due to the increased age of the population and the capacity of clinical staff to face more critical patients.
The last indicator considered in this section is EuroSCORE (ES). EuroSCORE (European System for Cardiac Operative Risk Evaluation) is a risk indicator for patients undergoing cardiac surgery, which has been widely used since it was introduced in 1999. For simplifying the use of the system and promoting the evaluation of the risk even in a time where information technologies were not so important, the ES was first published and largely clinically accepted as an additive system in which it was given to each risk factor a weight or a number of points that, when combined, provided an estimate of the expected percentage mortality for a patient undergoing a cardiosurgery procedure. However, because of its addictive nature, the additive ES underestimates the risk in some groups of patients. At the same time, there has been an exponential growth in the availability of information technologies for the cardiac and surgical units of hospitals, which explains why the use of a risk model based on the equation complete logistics was gradually established [18]. Analysis included in this paper concerns only logistic ES. Approximating ES values to integer numbers, Figure 7 represents the number of patients per every specific logistic ES value. Figure 7 shows that the majority of patients (about 75% of the whole population) falls into the first six ranges of ES.
Fig. 2 Number of patients for different intervention. 
Fig. 3 Overall number of device used each year. 
Fig. 4 Time trends of number of devices. 
Fig. 5 Yearly average LoS for different type of patients. 
Average LoS for each type of patients.
Percentage of patients being part of the represented age range.
Fig. 6 Percentages of patients in every considered age range. 
Fig. 7 Number of patients for every specific logistic ES values. 
3.3 Outcome
The outcome indicators analyzed in this paper are the exitus and the complications of the patients. Starting from the first one, Figure 8 shows the yearly mortality rate for two cluster of patients based on the different types of intervention:

no distinction between different types of intervention (Total);

patients with isolated CABG intervention (CABG).
Patients with isolated CABG intervention were considered, because it was well accepted as proxy of common cardiac surgery performance indicator. Figure 8 shows a decreasing trend of the mortality rate: the average mortality rate considering the whole population is 5.2%, while the other one considering only patients with CABG intervention is 2.2%. Analogue analysis of the mortality rate has been done considering only patients with LoS < 30 days, because this kind of indicator allowed making a comparison (not reported in this paper) between the ICU here considered and some other national or foreign ICUs. Data shows that the overall average mortality rate is 4.02%, instead, patients with LoS < 30 and with only isolated CABG intervention have an average mortality rate, of 1.90%. The analysis of the most important critical care quality indicators makes it necessary to discuss the these results (Tab. 3). In particular a comparison between the expected mortality rate (following only the logistic ES) and the real mortality rate should be done. According to clinical practice it was necessary to exclude, in this type of analysis, very critical groups of patients that are not involved in the ES calculation: these patients present specific interventions (VAD, BIVAD, Levitronix, ECMO, Biopump, Artificial Heart, Transplantation) or a very long Length of Stay (LoS > 30 days). First of all, in Table 4, the global difference between the real and the expected mortality is analyzed considering the whole population of patients. The observed mortality is lower than the expected of 1.14% suggesting the fact that the Unit has very successful performances. Secondly, in Table 5, the difference between the expected and observed mortality rate was analyzed, but now considering different ranges of ES in order to check the unit's performances in homogeneous groups of patients as much as possible. The table shows that the Unit mortality rate is better than the expected one in each range, and the unit's performances are good. Lastly, in Figure 9, the difference between the mortality rates is analyzed considering the observed mortality rate values for each unit of logistic ES (the real ES values of patients were approximated to the numeric unit in order to have 1%, 2%, 3%, etc.). Figure 9 shows that the performances of the unit are better than the expected until the group of patients with logistic ES 12%. After this value, the curve of the observed mortality fluctuates around the ideal curve. This is probably because the number of patients falling into these groups is too small to have a significance, and it is therefore difficult to explain the corresponding results.
The second outcome indicator concerns clinical complications of patients (i.e. bleeding, redo, and general neurological episode). Table 6 shows the number of patients having a complication (including isolated exitus).
Fig. 8 Mortality rate for different types of intervention, yearly analysis. 
Comparison of CABG and other surgery mortality rate.
Real and expected mortality rate.
Real and expected mortality.
Fig. 9 Logistic ES forecast ability of observed mortality. 
Number patients having a complication.
4 Statistical quality control in intensive care unit (ICU)
Another analysis of exitus has been made using the instruments of SQC, in particular the Control Chart which are usually used to study the stability of a process. Even in the study of health services the SQC techniques are useful to monitor the stability of performance indicators (as LoS and exitus) but also to evaluate their variability due to external factors and to process changes (e.g. introduction of new surgical or clinical protocols) [1–4]. It is well know that the Control Chart is a statistical tool used to distinguish between variations in a process resulting from common causes and variation resulting from special causes. One goal of using a Control Chart is to achieve and maintain process stability. Process stability is defined as a state in which a process has displayed a certain degree of consistency in the past and is expected to continue to do so in the future. This consistency is characterized by a stream of data falling within control limits. The most important use of a control chart is to improve the process.

Most processes do not operate in a state of statistical control.

Routine and attentive use of control charts will identify assignable causes. If these causes can be eliminated from the process, variability will be reduced and the process will be improved.

Control charts will only detect assignable causes. Management, physicians, nurses and engineering action will usually be necessary to eliminate the assignable causes.
4.1 Pchart
This attribute control chart is widely used for monitoring a process quality by the knowledge of the fraction (proportion) of non conformity. This fraction is defined as the ratio between the number of non conformity compared with the total number of the population. The analyzed set of patients may have several quality characteristics that are examined simultaneously. If the patient does not conform to standard on one or more of the characteristics, it is classified as non conforming. In this paper the characteristic analyzed is the exitus, the patient is non conforming when his exitus is positive. The statistical principles underlying the control chart for a non conforming fraction are based on the binomial distribution [10]. Therefore, considering a 3−σ interval the Control Limit and the Central Line are given by(1)
In the following exitus over the observed years are reported. For each year all the patients are taken and are computed into the sample fraction of non conforming p̂, and the statistic of p̂ is plotted in the chart. In this case of study it is assumed that p_{0} = 3.8%. As long p̂ remains within the control limits and the sequence of plotted points does not exhibit any systematic non random pattern, it is possible to conclude that the process in control at the level of p. Moreover, Pchart (Fig. 10) shows that the first year represents a statistically outcontrol process for exitus and successively the process is established is incontrol (excepted for 2009 data). Outcontrol process means that a special cause of variation has occurred. The decreasing trend itself is an indicator of the improvement of the Unit performances since the formation of the team.
Fig. 10 Pchart of exitus. 
4.2 Uchart
In other cases, the measure of quality does not consist in the monitoring of nonconforming products in output from the process, but in the calculation of the average number of nonconformities per unit of reference, therefore this attribute control chart (Uchart) assumes the possibility that each unit may present more than one event of nonconformity as opposed to the Pchart. This second approach involves setting up a control chart based on the average number of nonconformities per inspection unit. If an x total non conforming are found in a sample of n inspection units, then the average number of non conformities per inspection unit is:(2) x is a Poisson random variable; consequently, the parameters of the control chart for the average number of non conformities per unit are as follows:(3) where u̅ is the observed average number of non conformities per unit in a preliminary set of data. In this case the nonconformity is always the exitus, but the unit of reference is the number of days stay in hospital for all patients every year. In our case of study u_{0} = 1.5%. Ucharts (Fig. 11) shows that the exitus are outcontrol only in the first year that the team settled, and successively are not, showing again the improvement of Unit performance thanks to stabilized team skills.
Fig. 11 Uchart of No. exitus. 
5 Advanced data analysis
In order to deeply analyze the department have been introduced some derivatives proxies. Starting from the ES value the relation between the ES and LoS and between the ES and the discharge frequency has been analyzed.
ES is a method of predicting the chances of dying during or shortly after undergoing heart surgery. ES is a tool that was designed for physicians to calculate how risky a heart operation is and gives an idea of the likely risk if a group of patients have a similar operation and health status. All operations and other treatments have benefits and risks. It is important to emphasize that ES can only shows the risks and not the benefits. Although it is to be noted that patients exact scores can only be determined by someone who has access to all the detailed medical history [19].
ES is considered as a synthetic proxy for the admission health status of the patient and it is assumed that there exists a relation between the ES and the LoS that is considered to be a reliable and valid proxy for measuring the consumption of hospital resources [16]. It is considered for example that when two patients have similar operations, it is reasonable to assume that patient with lower ES – risk and less likely complications – will spend less time in hospital. In order to investigate this relation between LoS and ES, the patients are divided considering the ES and for each cluster the average LoS is evaluated. In Figure 12, xaxes represent the value of the ES and yaxes the average LoS, circle marks represent the average LoS estimated in the cluster, and x marks are the average LoS estimated using a linear regression. The coefficients estimated from the linear regression are β_{0} = 2.10 and β_{1} = 0.26, and the average absolute error (AAE) is 0.2056.^{1} The value of AAE suggests that there exists a linear relation between LoS and ES. The ES value is recorded when the patient is admitted therefore it is possible to estimate the expected LoS () by the following relation(4)
In Table 7, the details of the cluster with more then 200 patients are reported. For a larger value of ES, riskier patients, the standard deviations of the LoS increase, this behaviour can be explained considering that the ES is a proxy of the admission health state. If a patient has larger ES he has a lower health state therefore his hospitalization will be longer and more uncertain [20].
Using the ES values a model is also introduced to describe the discharge number of patients. This proxy can be used to provide information about the system's overall performance, including its abilities, efficiency, and effectiveness. The distribution of the discharge number of patients supplies the probability that a given number of patients is discharged in one day [21]. Each patient has a unique medical history and reacts differently to the treatment, it is impossible to quantify uniquely the health status after the same operation. Even if patients with the same ES are considered it is not possible to exactly forecast the future value of the health status [17]. Hence a proxy of the health status is introduced that gives some information on the probability distribution of the discharge number. Based on admission patient status ES the expected LoS trough relation (4) is evaluated. After 1 day of hospitalization, if the patient has not been discharged, his expected LoS decreases by 1 therefore(5) where v_{i,k} is the expected LoS of patients i at time k. If the department is occupied mainly by patients that have small expected LoS more discharges are expected. The value v_{i,k} represents the remaining expected LoS of patient i at time k nevertheless during some hospitalizations the value v_{i,k} could be less then zero, which means that, given the starting linear model, the patient should be already discharged.
Let A be defined as(6)where N_{k} is the number of patients hospitalized in time k. The value of A_{k} counts the number of patients that are expected to be discharged and depends on N_{k}. If the remaining expected LoS v_{j,k} for patient j at time k is less then 0 then the indicator function counts 1. Otherwise the indicator function returns a value of 0. According to model (4) exists a linear relation between the ES and the LoS therefore for hospitalized patients the expected time to discharge is recorded. Equation (6) only counts the number of patients having an expected time to discharge less then zero.
It is proposed to describe the discharge frequency using the A value. The number of patients having an expected LoS less or equal to zero could be an observable proxy of the discharge number. In this case it is assumed that the value of A_{k} will provide information on the discharge number of patient in time k. Data are clustered considering the value of function A in time k − 1 and for each cluster the average number of discharges is evaluated. Table 8 reports the average number of discharge and standard deviation . For each cluster the empirical distribution of u_{k} is modelled using a weighted average of a binomial distribution and a negative binomial distribution (see Appendix A.1). The binomial number of trial is the maximum number of patients that could have been hospitalized in the department (n_{max} = 13) and the negative binomial number of failures as the maximum value of A recorded (A_{max} = 5). The parameters estimation is performed using the minimum mean squared error (MMSE) method.
In Table 9, p_{b} gives the binomial success probability, p_{n} is the negative binomial success probability, ñ_{b} is binomial number of success, and ñ_{n} is the number of failures. Data shows that the proposed probability distribution model provides a good description of empirical frequencies.
In the end the relation between the LoS and number of patient is analyzed. LoS gives the measure of the efficiency of the process and it is used as the measure of costs of the ICU and of the global hospital. The probability distribution of LoS can be described using a statistical approach, nevertheless this is an endogenous variable and it is important to analyze its relation to other variables. To analyze the relation between LoS and number of patient in the department, the recorded patients are divided into subsets. The clusters are based on the average number of patients that stay in hospital during the hospitalization of a given patient, let this variable be Np (see Appendix A.2). This proxy represents the occupancy level of the department. In Figure 13, the average time spent in ICU for different clusters are reported; in the xaxis the cluster and yaxis are reported represent the average LoS. All analyzes are conducted on a cluster composed of at least 20 patients. Figure 13 shows that an increasing of Np is associated with an increased observed LoS until the Np exceeds a threshold value. It is assumed that its decline in LoS is related, at least in part, to a higher discharge rate. As could be expected an increase in the patient number leads to an increase of the LoS. Anyway after a fixed amount the LoS starts to decrease, this is because the ward is almost full and the admission demand forces the physician to discharge the patient early.
Fig. 12 Relation between average LoS and ES. 
Cluster details: is the average LoS, is the average LoS estimated by the linear model, and σ is the standard deviation.
Average number of discharge , deviation standard , and number of day in each cluster.
Discharge distribution parameters and MSE estimation using a binomial and a negative binomial weighted average probability density function.
Fig. 13 Relationship between Np and LoS. 
6 Conclusion
The conceptual framework proposed in this paper allows to analyze different aspects of public health system performance.
Distinctive quality indicators and their representation are proposed in order to monitoring the health system. The reduction of CABG interventions (passing from 48% to 40%), the increase of mechanical support, and the decrement of the average LoS for CABG intervention show structural evolution of the department.
The average age of patients treated and the decreasing mortality rate for each type of intervention confirms the overall structural improvement of the department.
The control chart is an instrument for describing, in a precise manner, exactly what is meant by statistical control. In many applications, it is used for online process surveillance. In this paper the use of two different Control Charts is proposed. This tool puts in evidence, that after a starting period in which the team is settled, also in this case, it is observed an improvement of the performance due to the standardization of the procedure. The charts show a time trend that could be explained by the evolution of the department. First year data highlights that the process was out of control and analysis confirm that the process improves in the following years. The tools allowed a reasoned characterization of the system.
The next step is the development of methods to study the causes to be addressed in the organization entities of the target health system. A linear relation between LoS and ES is investigated and it is proven that for small value of ES the mortality is better than the expected one. Moreover LoS analysis suggest that the optimal number of hospitalized patient is 5.
An innovative method is also introduced for estimate the probability distribution of the number of discharge patients based on the estimated health status of the hospitalized patients. It is feasible to possible to forecast the conditional distribution of the daily number of discharge.
Appendix
A.1 Binomial and negative binomial distribution
In probability theory, the binomial distribution with parameters n and p is the discrete probability distribution of the number of successes in a sequence of n independent Bernoulli experiments, each of which yields success with probability p. In general, if the random variable X follows the binomial distribution with parameters and p ∈ [0, 1] it is called X ∼ B(n, p). The probability of obtaining exactly x success in n trials is given by the probability mass function:(7)
The negative binomial distribution is a discrete probability distribution of the number of successes in a sequence of independent and identically distributed Bernoulli trials with probability q before a specified number of failures (denoted r) occurs. In general, if the random variable Z follows the negative binomial distribution with parameters r ∈ (0, ∞) and q ∈ [0, 1] it is called Z ∼ NB(r, q). The probability of obtaining exactly z success before r number of failures is given by the probability mass function:(8)
Hence the probability distribution used to describe the observed frequency of a number of discharges (U_{k}) given the value of A_{k−1} is a weighted average of a binomial distribution and a negative binomial distribution with the following probability density function(9)
A.2 Np variable
Considering a patient x that stay in hospital for 5 days. During his hospitalization in the department he is treated with other patients. For example the number of patients is reported in Table 10.
Number of patients in the department during the hospitalization of patients x.
Given this scenario it is reasonable to assume that patient x in average stay in hospital with other 3 patients and this variable is called Np.
References
 F. Clemente, M. D’Arco, E. D’Avino, The use of a conceptual model and related indicators to evaluate quality of healthcare in intensive care units, Qual. Eng. 26, 196–205 (2014) [CrossRef] [Google Scholar]
 Health Services Research Group (HSRG), Quality of care: what is quality and how can it be measured? Can. Med. Assoc. J. 142, 2153–2158 (1992) [Google Scholar]
 P. Winkel, N.F. Zhang, Statistical development of quality in medicine (Wiley, 2007), pp. 1–7 [CrossRef] [Google Scholar]
 B. Kemper, J. de Mast, Measurement plans for process flow improvement in services and health care, Qual. Eng. 25, 437–450 (2013) [CrossRef] [Google Scholar]
 M. Hensher, B. Keogh, Quality metrics (Elsevier Ltd., 2009), pp. 393–396 [Google Scholar]
 A. Donabedian, The quality of care: how can it be assessed? JAMA 260, 1743–1748 (1988) [CrossRef] [PubMed] [Google Scholar]
 A. Handler, M. Issel, B. Turnock, A conceptual framework to measure performance of the public health system, Am. J. Public Health 91, 1235–1239 (2001) [CrossRef] [PubMed] [Google Scholar]
 C. Herman, W. Karolak, A.M. Yip, K.J. Buth, A. Hassan, J.F. Lgar, Predicting prolonged intensive care unit length of stay in patients undergoing coronary artery bypass surgery – development of an entirely preoperative scorecard, Interact. Cardiovasc. Thorac. Surg. 9, 654–658 (2009) [CrossRef] [PubMed] [Google Scholar]
 L.I. Worthley, Quality control, audit, adverse events and risk in the intensive care unit, Crit. Care Resusc. 2, 304–307 (2000) [PubMed] [Google Scholar]
 D.C. Montgomery, Statistical quality control: a modern introduction (Wiley, 2009), pp. 3–68, 179–267, sixth ed. [Google Scholar]
 C.M. Lilly, S. Cody, H. Zhao, K. Landry, S.P. Baker, J. McIlwaine, M.W. Chandler, R.S. Irwin, Hospital mortality, length of stay, and preventable complications among critically ill patients before and after TeleICU reengineering of critical care processes, JAMA 305, 1777–1785 (2011) [CrossRef] [PubMed] [Google Scholar]
 Proposal for a COUNCIL DECISION establishing the Specific Programme Implementing Horizon 2020 – The Framework Programme for Research and Innovation (2014–2020) Brussels, 30.11.2011 COM (2011) 811 final, 2011/0402 (CNS). [Google Scholar]
 E.A. McGlyn, Identifying, categorizing, and evaluating health care efficiency measures (AHRQ, 2008), Vol. 8 [Google Scholar]
 M. De Vos, W. Graafmans, E.L. Keesman, G. Westert, P.H.J. Van Der Voort, Quality measurement at intensive care units: which indicators should we use? J. Crit. Care 22, 267–274 (2007) [CrossRef] [PubMed] [Google Scholar]
 S.M. Berenholtz, T. Dorman, N. Koni, P.J. Pronovost, Qualitative review of intensive care unit quality indicators, J. Crit. Care 17, 1–12 (2002) [CrossRef] [PubMed] [Google Scholar]
 P.H. Millard, Modelling hospital resource use: a different approach to the planning and control of health care systems (Royal Society of Medicine, London, 1994), p. 2937 [Google Scholar]
 L. Straney, A. Clements, J. Alexander, Quantifying variation of paediatric length of stay among intensive care units in Australia and New Zealand, Qual. Saf. Health Care 19, 1–5 (2010) [CrossRef] [Google Scholar]
 F. Roques, P. Michel, A.R. Goldstone, S.A. Nashef, The logistic EuroSCORE, Eur. Heart J. 24, 1–2 (2003) [CrossRef] [Google Scholar]
 T.S. Kurki, U. Hakkinen, J. Lauharanta, J. Ramo, M. Leijala, Evaluation of the relationship between preoperative risk scores, postoperative and total length of stay and hospital costs in coronary bypass surgery, Eur. J. Cardiothorac. Surg. 20, 1183–1187 (2001) [CrossRef] [PubMed] [Google Scholar]
 P. Loponen, M. Luther, J. Nissinen, J.O. Wistbacka, F. Biancari, J. Laurikka, H. Sintonen, M.R. Tarkka, EuroSCORE predicts healthrelated quality of life after coronary artery bypass grafting, Interact. Cardiovasc. Thorac. Surg. 7, 564–568 (2008), doi:10.1510/icvts.2007.174144 (Epub 2008 Apr 7) [CrossRef] [PubMed] [Google Scholar]
 M. Papi, L. Pontecorvi, Hospital resource consumption modelling, Res. Bus. Manage. 1 (2014) ISSN: 23308362 [CrossRef] [Google Scholar]
Cite this article as: Fabrizio Clemente, Marco Papi, Luca Pontecorvi, Antonio Menichetti, Evaluation of indices for the measurement of quality in health systems, Int. J. Metrol. Qual. Eng. 7, 405 (2016)
All Tables
Cluster details: is the average LoS, is the average LoS estimated by the linear model, and σ is the standard deviation.
Average number of discharge , deviation standard , and number of day in each cluster.
Discharge distribution parameters and MSE estimation using a binomial and a negative binomial weighted average probability density function.
All Figures
Fig. 1 Schematic of a model of ICU department that highlights as quality indicators a set of parameters related to the structure, process, and outcomes [1]. 

In the text 
Fig. 2 Number of patients for different intervention. 

In the text 
Fig. 3 Overall number of device used each year. 

In the text 
Fig. 4 Time trends of number of devices. 

In the text 
Fig. 5 Yearly average LoS for different type of patients. 

In the text 
Fig. 6 Percentages of patients in every considered age range. 

In the text 
Fig. 7 Number of patients for every specific logistic ES values. 

In the text 
Fig. 8 Mortality rate for different types of intervention, yearly analysis. 

In the text 
Fig. 9 Logistic ES forecast ability of observed mortality. 

In the text 
Fig. 10 Pchart of exitus. 

In the text 
Fig. 11 Uchart of No. exitus. 

In the text 
Fig. 12 Relation between average LoS and ES. 

In the text 
Fig. 13 Relationship between Np and LoS. 

In the text 
Current usage metrics show cumulative count of Article Views (fulltext article views including HTML views, PDF and ePub downloads, according to the available data) and Abstracts Views on Vision4Press platform.
Data correspond to usage on the plateform after 2015. The current usage metrics is available 4896 hours after online publication and is updated daily on week days.
Initial download of the metrics may take a while.