© C. Bezier et al., published by EDP Sciences, 2020

1 Introduction

In the face of the SARS-CoV-2 epidemic, on 25 January 2020, the World Health Organization (WHO) alerted the world to the necessity of rapidly implementing mass screening for SARS-CoV-2 via the real time reverse transcription PCR method, known for short as RT-qPCR [1]. Clinical laboratories are obviously in the front line when it comes to implementing this policy of mass screening and testing.

In France, clinical laboratories are inter-coordinated at regional level by Regional Health Agencies (Agences Régionales de Santé, or ARS) in order to ensure the efficient distribution of testing across the country. These laboratories answer to the ARS on matters of numbers of tests and collection-to-result turnaround time. In common with other health facilities, they also receive recommendations from the French National Authority for Health (Haute Autorité de la Santé, or HAS), as well as from the French Ministry of Health. The laboratories routinely communicate their results to the prescribers and, in certain cases, patients directly. They also transmit their epidemiological data to Public Health France (Santé Publique France, or SPF), which collects national data each day before sending it to the French Ministry of Health, thus enabling day by day monitoring of the national situation [2].

Faced with this epidemic and the abnormally high demand for diagnosis, the laboratories have had to implement brand new molecular tests, which creates huge challenges in terms of staff, equipment and logistics if the demands are to be met [3].

Adapting to this situation often forces clinical laboratories to:

• Reduce their usual activity so as to concentrate efforts on SARS-CoV-2 testing;

• Divert existing staff or recruit and train new staff to carry out the tests;

• Select, purchase, implement and validate new molecular tests for the detection of SARS-CoV-2 in a short space of time;

• Manage stocks and plan orders while facing reagent shortage. The worldwide testing policy has forced laboratories to rapidly acquire a lot of supplies of, notably, extraction reagents, creating a global shortage [4,5]. Due to the restrictions imposed by border closures and the slowdown in global trade, French laboratories have also had trouble ordering from suppliers outside France.

Given the urgency of the situation and the rapid change in configuration that clinical laboratories have had to put into effect, the time spent on the acquisition, validation and control of new processes has undoubtedly been insufficient. The risk, therefore, of erroneous results (false positives or negatives) is real.

According to the literature, the reliability limits of tests for SARS-CoV-2 by real-time RT-PCR derive, amongst others, from the choice of anatomical site for sample collection [5–7], the quality of the collection process, the sample collected and its transportation [8,9], the evolution of viral load over time [9,10], the performance of the detection test and PCR kits, particularly as regards sensitivity [11–13], the choice of molecular targets of the detection tool [3,8,9,14–16] and the genetic diversity of SARS-CoV-2 [17–19]. It is vital for laboratories to be aware of these limitations and adjust their practices in order to optimize accurate interpretation of real-time RT-PCR results.

Real-time RT-PCR (Reverse Transcription − real time Polymerase Chain Reaction) is a complex method comprising several stages. To ensure the reliability of test results, it is important to manage the risks that arise at each stage of the process, from first contact with the patient to the transmission of the diagnostic test report.

The twofold object of this article is to identify and analyze the principal risks encountered with the SARS-CoV-2 detection process by real-time RT-PCR and to suggest ways of dealing with them. The approach used for the first part is the Fishbone Diagram (5M method), as it offers rapid identification of potential risks. For the second part, the identified risks are analyzed in detail through Failure Mode, Effect, and Criticality Analysis (FMECA) and prioritized in terms of the threat they pose, a process that brings to light the corrective and preventive actions that are most urgently needed. The results of the analysis are discussed, and solutions are suggested for each risk.

2 Detection of SARS-CoV-2 by real-time RT-PCR and risk analysis

2.1 Molecular biology techniques in medical biology

PCR (Polymerase Chain Reaction) is a recent molecular biology technique, discovered by Mullis [20] in 1985, and which earned him the Nobel Prize in Chemistry in 1993. The principle of this technique is the amplification of specific DNA (Deoxyribo Nucleic Acid) sequences with the help of a DNA polymerase enzyme, which allows the synthesis of a complementary strand from a template strand [21].

Following the discovery, numerous methods derived from PCR have been developed, such as RT-PCR, the principle of which is shown in Figure 1 [22]. Prior to PCR, the conversion of a single stranded RNA (Ribo Nucleic Acid) into a double stranded complementary DNA occurs through the action of a different enzyme, a reverse transcriptase. This makes the PCR reaction possible with fragments of single stranded RNA.

The nucleic acids that are to be amplified can be found in the nucleus of cells or within virus particles and are therefore not directly accessible. An extraction stage of the nucleic acids is necessary first, which is achieved through chemical lysis of the membranes and proteins, making the nucleic acids accessible for synthesis reactions such as PCR [23].

Since its invention, PCR, in particular real-time PCR, has become extremely widely used in clinical laboratories, especially in the field of clinical microbiology. Compared with traditional methods of culture and identification of microorganisms and viruses, molecular biology offers alternatives that are fast, sensitive, specific and reproducible [24], and for various types of sample: urine, respiratory specimens, stools, blood, etc. It can rapidly detect pathogenic agents that are difficult to cultivate or observe (fastidious bacteria on ordinary culture media, or parasites and eggs that are not easily detectable under a microscope). In the case of viral diseases, PCR means that viral culture in cell lines, which is long, costly and of questionable reliability, can be avoided.

As with any particularly sensitive test method, PCR is very susceptible to the risk of contamination, which causes false positives. At the other end of the scale, the method is also very vulnerable to amplification inhibitors, which causes false negatives. To offset these risks, laboratory good practices must include the following with standard test series:

• A positive control: a known sample containing the target pathogen or target DNA or RNA fragment;

• A negative control: a sample without the target analyte, or ultra-pure water. In the latter case, the negative control is called a “blank”;

• An internal control: a known sequence, normally amplified using collected samples (human sequences or sequences deliberately added to the sample as a control). This is a means of checking the conformity of the PCR reaction and the absence of inhibitors before concluding a result to be negative [25].

PCR is also very sensitive to the quality of sample collected, which is to some extent linked to the quality of transportation and storage of samples, especially with RNA. PCR necessitates specific equipment and reagents, experienced specialist staff, clearly defined and presented operating procedures, and specific dedicated workspaces [24].

Finally, a PCR result must only be used within the limits of the information it offers: PCR allows the detection of the presence or absence of a target nucleic acid, but does not generally permit conclusions to be drawn as to the viability of a pathogen responsible for a pathology.

Fig. 1 Principle of RT-PCR with single stranded RNA [22].

2.2 Test process for the detection of SARS-CoV-2 by real-time RT-PCR

SARS-CoV-2 is an RNA virus, and for that reason detection of the virus today is accomplished primarily through a molecular biology approach, real-time RT-PCR, as described in Figure 1. The usual approach is a multiplex system allowing the detection of multiple targets.

Before analyzing the risks associated with the detection of SARS-CoV-2, it is important to look at the context of medical biology tests.

Medical diagnostic tests are carried out in three processes, in accordance with the ISO 15189 Standard [26]: pre-analytical, analytical and post-analytical. These three processes are successive and dependent: the quality of the input at each stage directly impacts the quality of the output. Each process, therefore, has the ability to affect the reliability of the analytical report established at the end.

A process mapping [27] for a clinical laboratory is illustrated in Figure 2. This overview of the different activities of this type of laboratory shows the diversity of jobs and missions (nurses to take samples, secretaries, technicians, biologists, etc.), all of whom directly or indirectly influence the internal and external quality of the process, and in fine the conformity of the test results (positive or negative), in relation to the actual status of the patient (infected or not). For this reason, a smooth and controlled operation from start to finish is essential, as is the fluid and reliable transmission of information relating to both the samples and the execution of the tests.

Each process comprises steps that must be carefully outlined if the step itself, what is at stake, the role and influence of the human participants and the equipment and materials are to be fully understood.

The pre-analytical process involves all the steps of sample collection, sample transportation and recording of patient files, up to but not including the start of the test phase. This is the process that involves the broadest range of specialist skills and trades (nurses who take samples, secretaries, etc.). The description of the pre-analytical process is provided in Appendix 1 of this article.

The analytical process involves the detection of SARS-CoV-2 by real-time RT-PCR from, typically, nasopharyngeal swab samples. The analytical process comprises the viral inactivation, extraction of nucleic acids, preparation of PCR mix, Reverse Transcription reaction (RT) then PCR (or amplification) and, finally, validation of results. The example used in our risk analysis is a PCR method comprising both manual and automated steps, with inactivated samples placed in ready-to-use extraction plates and undergoing automated extraction. In the example configuration, preparation and distribution of the mix are carried out manually, and the patient samples are also manually distributed in PCR strips that are specifically designed for thermal cyclers. The thermal cycler is linked to the LIMS (Laboratory Information Management System) of the laboratory, so the amplification results can be exported directly to patient files via unique case numbers. These files then require a technical validation followed by a biological validation before the diagnostic test report can be compiled and sent out. The description of this analytical process is shown in Figure 3. Any risk analysis is of course specific to the particular method implemented, but a certain number of risks are common to all comparable methods used to detect SARS-CoV-2 by means of PCR.

Finally, the post-analytical process concerns the communication of the results. This final stage plays a role in the external quality of the result (i.e. ability of the laboratory to provide a result that is compliant with the actual status of the patient: infected or not), in other words, the quality perceived by the stakeholders (prescribers and patients). The description of the post-analytical process is provided in Appendix 2.

Fig. 2 Process mapping of a Clinical Laboratory.

Fig. 3 Description of the analytical process: object of risk analysis.

2.3 Risk identification and analysis methods

According to the ISO 31 000 Standard [28], a risk is the effect of uncertainty, this effect being a deviation from what is expected. A risk is generally expressed in terms of risk sources, potential events, their consequences and their likelihood. A source of risk, in other words the cause of a risk, covers any factor or factors that, singly or collectively, can engender a risk.

In our assessment, a risk is the probability of an undesirable event being able to cause a non-compliant test result to be reported, or in other words:

• A result that does not correspond to the actual state of the patient: carrier or non-carrier of SARS-CoV-2 (false positives or false negatives);

• A result reported to the prescriber outside reasonable time limits: more than the statutory deadline of 24 h after sample collection, as indicated in Île-de-France Regional Health Agency (ARS) recommendations [29].

To be able to take into account all the different types of risk sources that exist, both technical and organizational, a transversal vision of all the activities of the clinical laboratory is needed. It is therefore essential for a laboratory to have a process approach.

In what follows, the analytical process defined above is the object of a risk analysis. For the purposes of the first part of this analysis, the Fishbone Diagram, also known as “5M Method” or “Ishikawa diagram” or “Cause and Effect diagram” has been applied to each stage of the process. This method is commonly used by the French National Authority for Health (HAS) for problem solving [30].

This method, based on a process approach, serves as part of the continuous improvement policy for quality in a laboratory as detailed in the IS0 15189 Standard [26], and facilitates implementation of the Deming PDCA method (Plan, Do, Check, Act) [31], the sequence of which is as follows:

• Identify potential sources of weakness or error of a system (a priori risk analysis);

• Develop plans for the implementation of improvements;

• Implement the action plan;

• Review the effectiveness of the action via a targeted review process or audit;

• Adjust the action plan and modify the system in accordance with results of the review and/or audit.

For the second part of the risk analysis detailed in this study, an in-depth analysis is made using FMECA (Failure Mode, Effects, and Criticality Analysis) [32,33], which is highly complementary to the 5M method: it allows the classification of identified risks in different categories, the distinction between different causes, the study of their effects and an assessment of their criticality relative to a criticality threshold set beforehand, and the proposal of corrective and/or preventive measures with a view to eliminating risks or reducing their criticality.

Each cause described is associated with its consequences (local effect followed by final effect) and a criticality calculated from three indices [34], in the following manner:

CRITICALITY = FREQUENCY × SEVERITY × NON-DETECTION

The frequency measures the number of occurrences of the cause versus the number of tests carried out. What need to be identified are the risks that can cause a non-compliant result to be reported, and the number of occurrences per number of tests expresses this quantity. In the case under study, it is assumed that average daily activity (Sundays and public holidays included) amounts to 500 tests per day.

The severity describes the magnitude of local and end effects. The severity is measured against three important criteria, all of which play a role in the internal and external quality of the process:

• Impact on the result. A direct consequence would be certain to provoke an erroneous result without any other condition being necessary (for example, misinterpretation of a positive or negative case). By contrast, an indirect consequence is an effect that may modify a result but that needs other conditions to be present for that to occur (for example, the risk of contamination of a sample);

• Loss of time and resources for the laboratory. The term “resources” makes it clear that each test represents a real cost for the laboratory that must be taken into account;

• The risk of a result being reported outside the defined turnaround time. Turnaround time is exceeded when outside the deadline of 24 h following sample collection recommended by the ARS.

Finally, non-detection of a failure mode takes account of our capacity to detect it. The higher the non-detection rating, the lower the likelihood or means of detecting the failure.

Each of these criticality categories is detailed in Table 1, which shows the FMECA rating chosen. The ratings are defined arbitrarily (generally based on experience) to prioritize them and allow the calculation of criticality.

The FMECA risk analysis thus makes it possible to prioritize risks identified, highlighting:

• Those with a maximum severity rating, in other words corresponding to category 5 on rating Table 1;

• Those with a high criticality that exceeds a given threshold, here arbitrarily set at 75.

The ratings shown in Table 1 and the criticality threshold can be modified or may evolve over time as experience is acquired and data accumulated in the laboratory.

3 Results

3.1 Risk identification with the 5M method

In order to describe the risks related to the analytical process comprehensively, the 5M method (Manpower, Mother-nature (Environment), Machinery (Equipment), Materials, (Method) has been used.

In the context of molecular biology diagnostics, “Manpower” refers to the laboratory technicians who carry out the test process, “Mother-nature (Environment)” refers to the technical workspace and environment in which the tests are conducted, “Machinery (Equipment)” refers to consumables, reagents, automated machines, etc., “Materials” refers to the samples taken from patients or the product of the preceding stage of the process, and “Method” refers to the organizational and optimization aspects of the laboratory diagnostic process.

First of all, the general risks are described, which are not specific to any particular analytical stage but concern the overall process and influence the reliability of the detection test for SARS-CoV-2 by real-time RT-PCR. These risks are shown in Figure 4.

Since each stage of the analytical process has its own characteristics and purpose, the risks specific to each of them have been considered separately. The stages in question are viral inactivation, extraction of nucleic acids, preparation of the PCR mix, amplification and validation of results.

As can be seen in Figure 5, the main source of risk at the inactivation stage is the contamination of samples (by virions from other samples, present in the environment or carried by the technician on laboratory coats, gloves, etc.), which can result in a false positive. The material (samples) may, however, be affected by identification problems, labeling and traceability.

At the extraction stage, to the issue of contamination are added the risks related to patient identification and the need for traceability throughout the analytical process, as shown in Figure 6. In addition, clinical laboratories can be faced with equipment and logistics issues: the use of an automatic device needs to be included in a flow study to ensure it does not end up delaying results. Moreover, these devices are subject to a real risk of breakdown. If a piece of equipment is essential but out of service, without the availability of a backup plan the breakdown renders the laboratory incapable of carrying out tests and reporting results.

Aside from the contamination and organization issues, the preparation of the mix is a crucial process that demands great rigor. Any technical error can compromise the PCR reaction, which in turn can invalidate the tests of an entire series of samples, an event that engenders significant cost and deadline problems for the laboratory. The risks related to the preparation of the mix are shown in Figure 7.

The amplification stage is also vulnerable to human error, even if it takes place inside a thermal cycler. The main risks are the conformity of the PCR program and the PCR strips (a critical consumable in this stage), and once again the matter of patient identification. What is more, a laboratory may have several thermal cyclers working in relay, a situation that requires careful flow and utilization management to exploit the full potential for testing without compromising the quality of results. All of these risks are shown in Figure 8. Finally, just as much as the extraction machines, the thermal cyclers need regular checks and maintenance in order to avoid technical failure.

The results validation stage marks the border between the analytical and post-analytical processes, being the final step before the transmission of results. It is a crucial stage in itself, while also a source of numerous risks, related to human error, in terms of patient identification, interpretation of results, etc. The analysis of results by a technician will trigger one of a range of significant actions for the sample, which must be carried out with rigor: report a result (positive or negative), retest, request a new sample, etc. These risks are shown in Figure 9.

The 5M method, as has been demonstrated, allows rapid identification of the risks related to the analytical process.

Fig. 4 Description of the risks related to the overall process using the 5M method.

Fig. 5 Description of the risks related to the viral inactivation stage using the 5M method.

Fig. 6 Description of the risks related to the viral RNA extraction stage using the 5M method.

Fig. 7 Description of the risks related to the mix preparation stage using the 5M method.

Fig. 8 Description of the risks related to the amplification stage using the 5M method.

Fig. 9 Description of the risks related to the results validation stage using the 5M method.

3.2 Risk analysis using the FMECA method

All the FMECA results based on the risks identified using the 5M method can be found in the table provided in Appendix 3 of this article. The analysis comprises 103 risks spread across the stages of the analytical process: viral inactivation, extraction of nucleic acids, preparation of the PCR mix, amplification and validation of results. Of the many risks studied, some are critical. The risks confirmed as critical, in other words those with a criticality that exceeds the threshold set arbitrarily at 75 or a severity at maximum (i.e. 5 on the rating scale), are shown in Table 2. These risks were analyzed for each stage of the analytical process in order to demonstrate the key principles of a FMECA analysis.

When all the risks are taken into account it becomes fairly obvious that critical risks are in the minority, since they represent only 18% of the total risks studied, as shown in Table 3. On the other hand, it should also be noted that each stage of the analytical process with critical risks still had residual risks that were unacceptable after the application of the measures recommended in this study. The risk analysis given in Appendix 3 reveals that the viral inactivation stage is not a major source of risks, since no critical risk was found among the 18 studied. It remains a key stage nonetheless from the point of view of staff biosafety. In the manual stages (inactivation, extraction and mix preparation), the risk related to method and organization is low: no critical risk observed and a maximum criticality of 64 established for the risk of contamination. They have an effect on the fluidity of the analytical process but do not directly influence the reliability of the result.

The risk of contamination is very present at every stage of the analytical process. On the other hand, the risks where the consequence is contamination do not have significant criticality. This is primarily due to the fact that the frequency of the undesirable events is low − not surprisingly, since the risk of contamination is well-known in clinical laboratories and a great deal of care is taken to avoid it. A number of preventive measures exist that effectively reduce the frequency of contaminations and, therefore, the criticality.

The nucleic acid extraction and mix preparation stages carry the highest number of risks. Moreover, each one has two residual risks that need to be dealt with again in order to bring them into line and make them acceptable. In addition, the validation of results stage has four critical risks:

• Series validated despite non-compliant IQC (Internal Quality Control);

• Misinterpretation of amplification curves and Ct;

• Procedure allowing manual entry of results and/or file numbers (risk of human error);

• Poor management of risk of human error (lack of traceability of the actions carried out, too little automation, etc.).

None of the risks related to this stage were able to be corrected by the measures suggested in FMECA. This stage of the process therefore needs particular attention.

3.2.1 Risks related to the extraction of nucleic acids

The criticality of the risks related to the extraction of nucleic acids is shown in Figure 10. For this stage, the causes that had a criticality above the threshold of 75 and a residual criticality that did not drop below that figure were those involving patient switching and poor sample quality. In fact, switching patients reached the maximum criticality observed of 100. As for the quality of the sample to be tested, this is one of the most essential conditions for a reliable result. Uncertainty related to the sample impacts on the reliability of the diagnostic test as a whole, which creates a high criticality, and notably a high risk of a falsely negative result. These are two critical risks, therefore, that require particular care and attention.

Fig. 10 Risks related to the extraction of nucleic acids and associated criticality.

3.2.2 Risks related to preparation of the PCR mix

The criticality of the risks related to preparation of the mix are shown in Figure 11. This stage contains numerous risks that equal or exceed the threshold of 75, notably when it comes to initial criticality.

The risk of switching patients is still present, and indeed has a maximum initial criticality of 100, and a residual criticality that is still 75. This is the same profile that was found at the extraction stage. Given that there are both manual and automated stages, involving numerous and repetitive operations, switching two samples or forgetting to pipette are both possible. In either of these events the impact is serious, with the real possibility of erroneous results, whether false positives or false negatives.

There are also risks due to what are called “technical” errors (problems of pipetting and bench organization): these are poor homogenization of the mix and samples, and PCR strips being left open for too long and causing a risk of contamination. Since the mix takes the form of a master mix that is distributed among a large number of samples, correct preparation of this mix, though not necessarily an easy matter, is crucial. Poor mix homogenization, for example, can inhibit the PCR reaction, meaning no interpretable results can be obtained for an entire series of samples, all of which need to be retested, and this obviously translates into extra costs and time loss for the laboratory. Meeting the turnaround time for testing is of critical importance.

Fig. 11 Risks related to mix preparation and associated criticality.

3.2.3 Risks related to validation of results

Four risks were identified at the validation of results stage, all related to human error. These were the interpretation of data (amplification and Ct (Cycle threshold) curves), poor IQC management, manual entry of results and/or file numbers and, in addition to these three, poor error risk control, all of which can have serious and direct consequences for a test result, and thus for a patient. In fact, these four critical risks all reach a maximum severity of five when an event occurs. However, if the criticality of these risks does not exceed 60, this is because the non-detection factor is low. In practice, LIMS and computerized laboratory management contribute considerably to the traceability of raw results and of actions carried out on files, making it possible to pinpoint a human error rapidly. Nevertheless, finding the error still takes time, and in that interval the technical validation may already have taken place, with a distinct likelihood of the diagnostic test report having been sent out. It is important not to be misled by these low criticalities, which are to some extent due to the limitations of calculating criticality through multiplication and the influence of the non-detection factor. It is important to take risks with a severity of 5, such as those noted here, seriously.

Finally, at several stages of the analytical process, critical risks due to machine issues were observed. As with any method involving automated stages, correct functioning of the process depends on correctly functioning equipment, in this case, the extraction machine and thermal cyclers, as well as ancillary equipment (refrigerators, freezers and Microbiological Safety Stations). If diagnostic equipment shuts down it is impossible to produce test results unless a back-up solution exists (reversion to an analogous method, provision of replacement equipment by the supplier, etc.) and can be deployed rapidly. In our investigation, machine failures represent critical risks.

4 Discussion and recommendations

Despite the volume of scientific literature available, there are few articles that deal in detail with the subject of risk analysis, central as that consideration is to the issue of testing for SARS-CoV-2 by real-time RT-PCR.

The 5M and FMECA methods used in this article are standard tools for a priori risk analysis and eminently suited to the investigation described, since they enable the comprehensive identification and analysis of the risks that exist, and at the same time allow their prioritization via an assessment of their criticality. Further methods (HACCP, APR, HAZOP, etc.) that are commonly applied in other sectors might be able to add other complementary or more specific facets to this study.

The evidence of the risk analysis suggests that the risks involved in SARS-CoV-2 detection by PCR are not fundamentally different from those encountered with other PCR tests carried out in clinical laboratories.

What the methodology presented in this article effectively does is to highlight the serious risks that need to be taken care of in the context of the clinical laboratory.

4.1 Influence of the human factor

The risks of human error linked to the manual entry of results and/or patient file numbers are very real. These risks can cause undesirable events such as the entry of an erroneous result (positive instead of negative or vice versa) or the input of a result in the wrong file and hence for the wrong person. To diminish these risks, which can have serious consequences for patients, it is important to keep manual data entry (of results, patient numbers, important information, etc.) to a minimum and opt for automated reporting wherever possible.

To limit human intervention in the management of patient files and data, and even in the interpretation of results, an IT tool is indispensable, all the more so when very large numbers of samples are to be tested. This IT tool encompasses:

• A barcode scanning system (with codes specific to the patient or the analytical process stage) in order to avoid having to enter file numbers, which can be long and thus a source of error risk;

• The availability of a good connection between the machines and the LIMS, with raw results transferred automatically for appraisal, in order to avoid manual entry of results or file numbers. Once again, if a low number of samples is handled, manual entry of results can be envisaged but should, for example, be subject to double checking by two separate technicians;

• The creation of computer settings to enhance interpretation of the Ct (cycle threshold) values once the curves have been analyzed by the technician. It is nevertheless essential for amplification curves to be analyzed one by one first, because, as mentioned above, their appearance plays an important role in the interpretation of results, as well as the Ct value of a sample. The practice in the laboratory thus benefits from a result validation process that is optimized, with rules of expertise to facilitate the interpretation of results, the handling of a positive case, or the handling of an invalid case needing to be retested. Traceability is needed for every action, on every tube and every patient file. Nevertheless, it is important to maintain a balance so as to implement a solution proportional to the size of the task (volume of samples to be treated) and the organization of other laboratories (Information Technology and human resources available).

The interpretation of results, both of series controls and patient results, should be carried out in accordance with a decision flowchart in order to harmonize practices and keep human error to a minimum [35]. This flow chart should mention all possible cases, number them, and define the appropriate action to be taken by the technician for each one: for example, “Report a positive (or negative) result”, “Send sample for retest”, “Controls non-compliant: series to be retested”.

The result validation stage is a crucial part of the test process. It is the final stage before the definitive transmission of results to prescribers and patients in the form of diagnostic reports.

An increase in the overall activity of clinical laboratories necessitates the deployment of additional material, but also human resources. Apart from the need for a large number of nurses to carry out sample collection at different sites, the implementation of a new test method often obliges laboratories to recruit new staff. For this reason, laboratories need to give thought to a specific staff training program to meet the new need, which can then become an integral part of the Quality Assurance system. The WHO, in its Laboratory Quality Management System manual, also makes recommendations for the training and qualification of laboratory staff [31].

The increase in team sizes, organizational changes and new work patterns (round-the-clock, seven days a week) create a need for fluent, continuous and efficient communication to facilitate the numerous and often urgent decisions that need to be made. It is also helpful to hold frequent brainstorming sessions to facilitate tactical or operational decisions, in other words those that affect the organization and efficiency of resources. Certain decisions may be taken without the presence of every individual concerned, but there should be adequate means of communication to ensure any team-members affected are informed, via an instruction book, a whiteboard fitted to a wall, emails, etc. Finally, a flow chart for the resolution of problems, indicating the corrective and preventive actions that need to be implemented, should also be created as part of the laboratory's Quality Assurance system. The main aim is to harmonize practices so as to limit inter-operator uncertainty and formalize practices so they match what has been discussed and agreed.

4.2 Risk of false-positives and contamination

A number of scientific articles highlight the risk of contamination during detection tests for SARS-CoV-2. The question of sample contamination is ever-present, as with any highly sensitive molecular biology process. Quantification of SARS-CoV-2 RNA per ddPCR in the work environment dedicated to the detection of SARS-CoV-2 has revealed the presence of RNA of the virus in the following proportions: up to 37.4 copies/cm² on workers' gloves; 26.25 copies/cm² on refrigerator door handles; 22.16 copies/cm² on goggles; 19.95 copies/cm² on the external surface of the centrifuge, not to mention on walls and so on [36]. Therefore, incidences of sample contamination by viral RNA (or, more realistically, PCR products in the form of double stranded DNA, which is more stable in the environment) at the time of mixing or during other steps are probable, with the direct consequence on the result at the end of a false positive.

Besides, primers and probes reagent could be a source of contamination and false-positive results. Positive results (with Ct value close to 22 or even 17) can occur with negative samples, because of contaminations of reagents [37]. The synthesis of long oligonucleotides containing SARS-CoV-2 sequences, often used as positive controls, have contaminated reagents (primers and probes) distributed to laboratories. This impacts the predictive value (POD) of a positive test. In this study, the diagnostic specificity reached 97.56%. With a prevalence of infection of 9.00%, the predictive value of a positive test was 0.802, which represents 20% false positives among the positive results obtained. As the prevalence of infection decreases, the POD ratio increases. In fact, when the prevalence of infection reaches 2.7%, the predictive value of a positive test is only 0.5319, i.e. almost half of false positives among the positive results obtained [37]. Among other measures, this article recommends checking each new batch of reagents, with negative controls, to ensure that they are not contaminated. Obviously, all potentially false positive samples should be re-tested, if possible, by another laboratory [37].

Contamination control is therefore essential to avoid the risk of error. It might, for example, involve the carefully thought-out organization of the laboratory benches used for mix preparation in such a way as to avoid the contamination of cones, pipettes, supports and samples. As well as good spatial organization, good temporal organization is another means to avoid leaving reaction tubes or strips open longer than necessary.

The WHO, moreover, recommends at least three physically separate rooms for the PCR operation [38]:

• PCR master mix preparation room;

• A room for the extraction of nucleic acids and addition of the DNA or RNA template;

• A room for amplification and manipulation of the amplified product.

Ideally, the workflow is unidirectional from clean areas (which are the first stages) towards the nucleic acid amplification room, a principle described as “forward motion”. In this way, only samples and operators move from one room to another, with specific precautions taken, such as changing laboratory coats. In the event of breaking this rule, any movement in the opposite direction requires a thorough disinfection process prior to entering an area considered to be clean [38]. Experience shows that implementing this type of protocol often calls for a reorganization of the workspace, to avoid having to convert portions of corridor away from their usual function to fit into the new scheme, and so that traffic through dedicated areas is completely controlled and regulated.

The formation of aerosol from contaminant products (RNA or DNA) means that practices limiting its formation must be adopted, which includes the quality of the pipetting. For the performance monitoring of the diagnostic process, the systematic use of blanks for every series and at every stage of the process guarantees early detection of a possible contamination and can even identify its origin. An “extraction blank”, composed of UltraPure™ RNase-Free water for molecular biology, in the place of a sample at the extraction stage makes it possible to eliminate any hypothesis of contamination at this stage. The blank should exhibit no amplification whatsoever (even of the internal control target, which sometimes amplifies human sequences or all RNA in a non-specific way). However, any contamination of the blank by human cells or microorganisms, which is possible, can be observed by amplification of the internal control target of the process (and not the specific SARS-CoV-2 targets). In practice, using a new pack or batch of ultra-pure water soon restores the integrity of the blank. It should be remembered that this consumable is particularly subject to contamination due to the fact of being used multiple times.

A “PCR blank”, containing only PCR mix and no nucleic acid, makes it possible to eliminate the hypothesis of contamination at this stage related to the PCR mix. As with the extraction blank, this control should exhibit no amplification whatsoever, even of the internal control substance.

To avoid contamination, it is essential to regularly disinfect laboratory benches, pipettes, supports and other equipment with a nucleic acid degradation product. The adoption of essential practices and good habits will be all the more successful if they are clearly set out in a comprehensive procedure and documented via maintenance traceability.

The process reliability limits can be improved through strict management of sample retests: in the absence of compliant amplification curves or in the event of a doubtful result, sending samples back for a retest can confirm (or otherwise) the initial result.

The nature of most PCR methods and kits on the market also helps reduce the risk of erroneous results by offering multiplex amplification aimed at two or even three SARS-CoV-2 molecular targets. In fact, the WHO stresses the importance of amplifying at least two molecular targets in order to increase the sensitivity of the test [39]. Even with this stringency, doubtful cases do arise, in particular when there is a conflict between the results obtained for each target. To deal with this eventuality, the laboratory needs to have established criteria for the definition of “doubtful cases”, and these cases need to be subjected to a retest in order to arrive at an interpretable result.

Practical experience in laboratories has shown that amplification curves that were initially non-compliant were replaced by readable plots after a second test, and that this was due to low initial efficiency of the PCR. The “doubtful” cases (amplification of only one out of two molecular targets, for example, curves with a non-standard shape, or late Ct values) are either positive after the second pass or are again doubtful. In both cases, the doubtful results were undoubtedly due to a low viral load in the sample, which gave late Ct values. In this situation, it is very important not to consider these results as negative. If a result is doubtful several times in a row, a new sample may have to be requested in order achieve a reliable diagnosis.

4.3 Management of reagents, consumables and critical equipment

To meet the growing demand for tests, many clinical laboratories have had to acquire new reagents and PCR kits for the detection of SARS-CoV-2. These kits and reagents have to be stored in accordance with the recommendations of the suppliers in order to ensure the product performance described by them.

To achieve this, an efficient system for the management of stock is essential, and this also forms part of the Quality Assurance system that the WHO recommends for molecular biology methods [38]. Up-to-date stock lists need to be maintained, as well as the storage requirements provided with the kits and reagents. However, this management system is not easy to maintain when no-one knows the usual delivery times (if dealing with a new supplier) or can forecast consumption in view of changes in demand. The management of stock to begin with is to some extent trial and error, until the balance between under stock and overstock can be established. General guidelines for the stock management policy of a laboratory do exist, however, notably in the Laboratory Quality Management System manual published by the WHO for clinical laboratories [31].

When it comes to critical machines (extraction devices, automated pipette machines and thermal cyclers), except in the case of planned maintenance outages, the whole process can come to a standstill if they breakdown for any reason, causing considerable delays and almost inevitable risk of deadline failure. For this reason, it is very important to anticipate this eventuality with the provision of a back-up method in case of problems with the normal machines and, if necessary, be prepared to subcontract certain stages to other laboratories.

As was stated in the section above about the influence of the human factor on the reliability of the analytical process, automation (or something at least approaching it) offers advantages when it comes to the practical organization of tests. This can take the form of:

• Standardized automated plate plans with the full name of the patient, their file number, the associated barcode, its position in the series and the corresponding series number. The data sheet is stored in a computer to ensure the traceability of each test series, is printed out and accompanies the series through every stage of the process up to and including validation of results, and is a way to control the risk related to patient identification by avoiding manual entry through the scanning system;

• Recourse to any automated approach, large or small, such as, for example the use of multichannel pipettes, which enable the physical activity to mirror the standardized plate plans.

4.4 Internal and external monitoring of the test process

For early and reliable detection of a disease, the analytical sensitivity of the diagnostic test must be optimal. To ensure this, it is vital to have access to the individual and comparative performance characteristics of the PCR kits available, and in particular their LOD (Limit of Detection) values.

It is also important to monitor the performance of the PCR kits in use throughout their life cycle and across different batches. The way this is done is to plot the Ct values (Cycle threshold) of the IQC obtained from positive and negative controls for each series, which define the limit values to be used as a reference for each test series. With this approach a potential deviation of the PCR system and, more generally, of the analytical process can be detected [40]. These values may, for example, be expressed in the form of a Levey-Jennings chart, used in conjunction with the Westgard rules [41]. Generally, decision rules are defined to adjudicate when a deviation is detected in a system, and to set out actions that will contribute to the reliability of the process during the term of its use.

Moreover, among the IQC used, it is a good idea to use a positive control with a very low viral load (close to the detection limit of the method) in order to confirm that the PCR technique is capable of detecting viral loads that are low but still above the defined detection limit. The absence of amplification in this detection-limit positive control indicates that the samples in the series being analyzed need to be retested in order to eliminate any risk of false negative results.

The quality assurance processes in place in clinical laboratories helps provide guarantees with regard to the identification and management of specific risks. But the very high societal expectation of receiving reliable results and of transparent processes makes external assessment of the organization of the laboratory a virtual necessity. For this reason, in addition to the internal monitoring process, a laboratory should be analyzed by external assessors, thus benefitting from an outside perspective on the quality of the testing process as well as from suggestions for other measures for improvement. Inter-laboratory comparisons, which are an integral part of the validation of the method and of good laboratory practices, also contribute to this external assessment of laboratory practices, while, ultimately, fostering improvements in and harmonization of the technical practices used for the detection of SARS-CoV-2 for the participating laboratories. It is at this point worth mentioning the works of Görzer et al. [42], and the results of an external assessment of 52 clinical laboratories, which included remarkable diversity of nucleic extraction kits as well as diagnostic kits. The fact that this work was carried out at the request of the Austrian public authorities and in consultation with the Austrian national accreditation body further underlines the importance of these external assessments.

4.5 Sample quality

Besides the analytical process, which is the main object of this risk analysis, many aspects of the pre-analytical process significantly condition the final test result. The quality of any process depends directly on the quality of the output data of the process preceding it. It is evident, therefore, that the slightest uncertainty related to the quality of a sample will have repercussions on the analytical process.

In the case of tests to detect SARS-CoV-2, a number of studies describe the sensitivity limits of the nasal swab method [8,9]. These are dependent on the quality of physical penetration during sample collection, which may not probe deep enough or may be of too short duration, to the point where it can be impossible to detect and isolate the virus in a person who is nonetheless a carrier. In this case, reporting a false negative result is highly probable. It is worth noting that nasopharyngeal sample collection is described as potentially painful, at the very least unpleasant, both physically and psychologically for those experiencing it, even with an experienced nurse [43].

Practical experience of screening has demonstrated that, for a vulnerable population such as the residents of a retirement home, the fear of physical pain mingles with confusion and apprehension at the arrival of sample-takers from the laboratory. These are outsiders, unfamiliar to them, not to mention the fact that they are dressed in coverall suits and have come to do a job the residents have probably never had done before. Understandably, the residents might feel anxious and perhaps threatened. In this situation, the following problematic situations can arise:

• Simple and categorical refusal to allow the sample to be taken. In this event, no-one can force the person to go ahead with the test;

• Abrupt gestures from the person, which can be a hazard to themselves and to the sample-taker;

• Tenseness, making it impossible to reach the furthest part of the nasal passages, which is necessary for a quality sample collection;

• Inability to take a sample from the second nostril if there is a need.

To minimize these difficulties, good training for sample-takers from qualified staff who know the local anatomy of the upper respiratory tract is essential [43].

Also, in order to harmonize practices, the clinical laboratory should have a sample collection manual that gives clear and easy to follow instructions, backed up by the recommendations in force, such as those published by the CDC (Centers for Disease Control and Prevention), for example [44].

The second aspect of the pre-analytical process concerns the transportation and storage conditions of the samples, which considerably condition the quality of the samples when presented for analysis.

Samples should be taken to the laboratory without delay in order to keep deterioration to a minimum. If the virions degrade and no longer protect their RNA, which is very unstable in the environment, the RNA may become inaccessible or degraded, rendering it non-amplifiable by PCR and increasing the likelihood of a falsely negative result. The CDC recommends that samples reach the laboratory as quickly as possible to ensure rapid storage between 2 °C and 8 °C. At this temperature, they can be kept for 72 h following collection. Beyond that timescale, the samples should be stored at −18 °C at least, or ideally −70 °C or less, according to the recommendations of the CDC and the WHO [39,44]. The WHO recommendations also state that freeze-thaw cycles should be avoided. The storage phase is all the more important in the light of evidence that longer storage at 4 °C (several days) of samples with a low viral load can be responsible for an increase in Ct during real-time RT-PCR and can, therefore, alter the status of the sample [45]. With a great many detection methods, a result is considered positive when its Ct are below a predefined threshold. The Ct for a sample taken from a person who is a carrier of SARS-CoV-2 yet incorrectly stored may be outside the defined limits and thus give rise to a false negative result. In practice, it can be difficult for clinical laboratories to comply with ideal transportation and storage conditions: collection is a continuous process carried out throughout the day, with samples stored in disinfected boxes, which are in turn stored in ice boxes, and then taken to the laboratory in the evening after as long as 10 h. This is due to the logistics of sample collection, and a lack of couriers and regular trips to allow the transportation of samples to the laboratory in a shorter time. Where transportation and/or storage conditions have not been optimal, extra attention should be paid to the results from the samples involved, with verification of the successful amplification of the internal control (indicating that the nucleic acids in the sample were still intact), that the Ct is not exceeded, etc. Caution and discernment need to be exercised and the result judged in the light of other clinical data, so as not to draw inappropriate diagnostic conclusions [6,9,46]. A detailed analysis of the results may lead to a decision to proceed with a thorough retest, in the expectation that a second pass will provide additional information regarding the presence or otherwise of SARS-CoV-2 in the sample.

5 Conclusion

This article presents an analysis of the risks of real time RT-PCR SARS-CoV-2 testing methods and of the reliability of the results thus obtained. Two methods were used: Fishbone Diagram (5M method) and FMECA (Failure mode, effects, and criticality analysis). The 5M method allows rapid identification of the risks that impact on the conformity of results of SARS-CoV-2 diagnostic tests. FMECA is highly complementary to the 5M method in adding detail to the findings and allowing the distinction and categorization of the different causes associated with the risks, as well as an assessment of their criticality.

While the methodology presented here is intended as a contribution to the scientific research into risks related to the diagnosis of SARS-CoV-2 by PCR, it can be used for other test methods and also serve as part of the Quality Assurance of a laboratory.

In this present instance, the risk analysis of the SARS-CoV-2 test process was conducted so as to take into account the interactions between the different intra-process (analytical process) and inter-process stages (pre-analytical, analytical and post-analytical). For example, the role of sample collection and management of reagents has been taken into account in the investigation of risks.

The investigation shows that the principal risks that impact on the conformity of the results of the diagnostic test concern the extraction of nucleic acids, the preparation of mix, and the validation of results. For the extraction of nucleic acids, high criticality risks (exceeding the threshold set from experimentation) are the risk of error when depositing samples on the extraction plate and sample non-conformity. For the preparation of the mix the highest risks are a non-homogenous mix and, predominantly, errors when distributing samples on the amplification plate. When it comes to the validation of results, the criticality is increased by the presence of a maximum severity index. The associated risks requiring particular attention concern the interpretation of test data, poor IQC (Internal Quality Control) management and manual entry of results and/or file numbers.

Recommendations are made to diminish the influence of the human factor, the risk of false-positives and internal contamination in the laboratory, the influence of the management of reagents and other consumables and critical equipment, as well as the influence of a poor sample. Attention is also drawn to the need to monitor, both internally and externally, the performance of the test process within a clinical laboratory in terms of quality and reliability.

The results and recommendations included in this article are based on lessons learned from the actual experience of a real clinical laboratory that carries out diagnostic tests for SARS-CoV-2 by RT-PCR. It is perhaps worth pointing out that a number of our conclusions on the risks and reliability of tests are shared by others. A literature review on the reliability of PCR tests is proposed to you by Bezier et al. [47] in addition to this article.

The reliability limits of diagnostic tests for SARS-CoV-2 are a reminder that a diagnostic result should not be interpreted on its own but viewed within the broader clinical context, taking into account symptoms, the medical state of the patient (chronic diseases, etc.), and thoracic CT (computerized tomography) scan imagery. If a negative result is obtained from a sample from the upper respiratory tract whereas there is serious suspicion of COVID-19, the WHO recommends laboratories to take other samples, notably from the lower respiratory tract, if that has not already been done.

Appendix 1

Appendix 1 Description of the pre-analytical process

Appendix 2

Appendix 2 Description of the post-analytical process

Appendix 3

References

All Tables

All Figures

	Fig. 1 Principle of RT-PCR with single stranded RNA [22].
In the text

	Fig. 2 Process mapping of a Clinical Laboratory.
In the text

	Fig. 3 Description of the analytical process: object of risk analysis.
In the text

	Fig. 4 Description of the risks related to the overall process using the 5M method.
In the text

	Fig. 5 Description of the risks related to the viral inactivation stage using the 5M method.
In the text

	Fig. 6 Description of the risks related to the viral RNA extraction stage using the 5M method.
In the text

	Fig. 7 Description of the risks related to the mix preparation stage using the 5M method.
In the text

	Fig. 8 Description of the risks related to the amplification stage using the 5M method.
In the text

	Fig. 9 Description of the risks related to the results validation stage using the 5M method.
In the text

	Fig. 10 Risks related to the extraction of nucleic acids and associated criticality.
In the text

	Fig. 11 Risks related to mix preparation and associated criticality.
In the text

	Appendix 1 Description of the pre-analytical process
In the text

	Appendix 2 Description of the post-analytical process
In the text