Consider moving average quality control when internal control is insufficient or inefficient – the time is now!
by Dr Huub H. van Rossum
Recently, significant improvements have been made in understanding and applying moving average quality control (MA QC) that enable its practical implementation. These include the description of new and laboratory-specific MA QC optimization and validation methods, the online availability thereof, insights into operational requirements, and demonstration of practical implementation.
Introduction
Moving average quality control (MA QC) is the process of algorithmically averaging obtained test results and using that average for (analytical) quality control purposes. MA QC is generally referred to as patient-based real-time quality control (PBRTQC) because it is one of various methods (e.g. limit checks, delta checks, etc) that use patient results for (real-time) quality control. MA QC was first described over half a century ago as ‘average of normals’ [1]. Since then, it has evolved into a more general MA QC concept not necessarily based on using mean calculations of the obtained ‘normal’ test results [2]. Although MA QC has been available for a few decades, its adoption by laboratories has been limited due to the complexity of setting up the necessary procedures, operational challenges and a lack of evidence to justify its application and demonstrate its value. During the past 5|years, however, significant improvements have been made in the field of MA QC, and research studies have addressed all these issues. Consequently, true practical application of validated MA QC procedures to support analytical quality control in medical laboratories is now possible. Furthermore, the recent improvements may well change the way we perform daily analytical quality control in medical laboratories in the near future.
MA QC optimization and validation
The recent significant improvements in the field of MA QC include, first and foremost, the description of new methods to design and optimize laboratory-specific MA QC procedures and to enable validation of their actual error-detection performance [2–5]. These methods use realistic MA QC simulations based on laboratory-specific datasets and thus provide objective insights into MA QC error detection [2]. To enable practical implementation, the requirement that the number of MA QC alarms must be manageable is now acknowledged as essential and has been fulfilled when setting up MA QC [2, 6]. The newly developed methods use a novel metric to determine the error-detection performance: that is, the mean or median number of test results needed for error detection. One of the new methods presents these simulation results in bias-detection curves so that the optimal MA QC procedure can be selected, based on its overall error-detection performance [5]. An example of a bias-detection curve and its application is presented in Figure 1. After selecting the optimal MA QC settings, an MA validation chart can be used to obtain objective insights into the overall error-detection performance and the uncertainty thereof. Therefore, this chart can be seen as a validation of the MA QC procedure. An example of an MA validation chart is presented in Figure 2 and shows that the MA QC procedure will almost always (with 97.5% probability) detect a systematic error of −4% (or larger negative errors) within 20 test results.
Importantly, this method has become available to laboratories via the online MA Generator application, enabling them to design their own optimized and validated MA QC procedures [7]. Laboratories can now upload their own datasets of historical results, study potential MA QC settings using this simulation analysis and obtain their own laboratory-specific MA QC settings and MA validation charts. Several laboratories have demonstrated that this tool has enabled them to obtain relevant MA QC settings and thus implement MA QC [8, 9].
Integration of MA QC with internal QC
Measurement of internal quality control (iQC) samples is still the cornerstone of analytical quality control as performed in medical laboratories. For many tests, iQC alone is sufficient to assure and control the quality of obtained test results. For some tests, however, iQC itself is insufficient. The reasons for this are related to certain fundamental characteristics of iQC that include: lack of available (stable) control materials, its scheduled character, the risk of using non-commutable control samples and tests with a sigma metric score of ≤4. For several reasons, PBRTQC or, more specifically, MA QC is a particularly valuable and powerful way to support quality assurance in such cases.
First, if no (stable) QC materials are available it is impossible, or it becomes complicated, to use iQC. This is, for example, relevant for the erythrocyte sedimentation rate, serum indices or hemocytometry tests including erythrocyte mean corpuscular volume in particular. MA QC is possible as long as patient results are available. Second, the scheduled character of iQC becomes a limitation and a risk when temporary assay failures or rapid onset of critical errors occur between scheduled iQC. Because a new MA QC value can be calculated for each newly obtained test result, MA QC can be designed as a continuous and real-time QC tool. In this context, detection of temporary assay failure by MA QC between scheduled iQC has been demonstrated for a sodium case [10], and several examples of MA QC detection of rapid onset of critical errors have been published for both chemistry and hematological tests [11]. Third, because PBRTQC methods such as MA QC use obtained patient results, by design there is no commutability issue. Fourth, and finally, for some tests iQC is intrinsically limited in its ability to detect relevant clinical errors, due to the low ratio of biological variations to analytical variations, as reflected in low sigma metric values. Such tests require frequent iQC analysis and application of stringent control rules. However, even with such an intensive and strict iQC set-up, the probability of detecting clinically relevant errors remains limited [12]. In contrast, MA QC has the best error-detection performance for tests with a low sigma value [13].
For all these reasons, MA QC is ideal for supplementing analytical quality control by iQC. Recently, an approach was presented that integrated MA QC into the QC plan when iQC was found to be insufficient [9]. This approach was based on first determining whether one of the abovementioned iQC limitations applied to a test. If so, then iQC alone was considered insufficient and MA QC was studied, using the online MA Generator tool (www.huvaros.com) to obtain optimal MA QC settings and MA QC procedures to support the analytical quality control [7, 9]. The MA QC error-detection performance was validated using MA validation charts. These latter insights into MA QC error detection also enabled iQC measurements to be reduced. The MA QC procedures alone provided significant error-detection performance, so running iQC measures multiple times a day would add only limited error-detection performance. Therefore, it was decided to run the iQC only once a day and add the obtained MA QC procedures to the QC plan.
Others have taken this a step further and studied MA QC not only for tests with limited iQC performance but also for a much larger test selection, in order to reduce the number of iQC measures and more efficiently schedule and apply iQC [4]. This approach has been shown to be successful for a large commercial laboratory with high production numbers. Since the MA QC error-detection performance improves with an increasing number of test results and benefits from a small number of pathological test results, this approach may be particularly valuable to the larger commercial laboratories. For such an approach, the key is objective insights into the error-detection performance of MA QC procedures such as obtained using MA validation charts.
Implementation and application of MA QC for real-time QC in medical laboratories
The final aspect in which there have been significant improvements in recent years relates to the practical application of MA QC in medical laboratories. Recently, an International Federation of Clinical Chemistry and Laboratory Medicine working group was founded that summarized medical laboratories’ experiences of practically applying MA QC and formulated several recommendations for both MA QC software suppliers and medical laboratories that are working on, or are interested in, implementation of MA QC [14, 15]. Also, a step-by-step roadmap has recently been published to enable MA QC implementation [9]. The first two steps of this roadmap – i.e. selection of tests and obtaining MA QC settings for them – were discussed in the previous two paragraphs.
The next step would be to set up and configure the software used to implement MA QC in medical laboratories. If you are interested in applying MA QC in your laboratory, it is important to review the available software (e.g. analyser, middleware, LIS, third party) and to decide which will be used to run and apply MA QC. Your decision depends not only on the availability of suitable software in or for the laboratory, but also on the actual MA QC functionality present in the software packages.
The minimum software features that are necessary to enable practical implementation have been formulated [2, 15]. In my view, key elements would be that the software supports: exclusion of specified samples (non-patient materials, QC results, extreme results, etc), calculation of relevant MA QC algorithms, applying SD-based as well as non-statistical control limits (plain lower and upper control limits), proper real-time alarming and – depending on the MA QC optimization method – presentation of MA QC in a Levey–Jennings or accuracy graph. Figure 3 presents an example of MA QC in an accuracy graph as operated for real-time QC in my laboratory. To enable effective implementation of MA QC, all of these software features should be configured.
The final implementation step I wish to address here is the design of laboratory protocols for working up MA QC alarms, which determines the extent to which an error detected by an MA QC alarm is acknowledged. An important requirement is that all MA QC alarms should be worked up by means of this protocol.
As previously indicated, because MA QC can generate many more QC results and alarms than iQC, a critical requirement of every MA QC procedure is a manageable number of alarms. As a result, when an MA QC alarm occurs there is a reasonable chance of detecting error.
A first common action as part of the MA QC alarm protocol would be to run iQC. This provides a quick insight into the size of the error and enables rapid confirmation of large errors. As a second step, re-running of recently analysed samples (in addition to running iQC) enables temporary assay failures to be detected and can confirm or exclude errors not necessarily detectable by iQC. Also, finally, a review of recently analysed test results to identify a pre-analytical cause or a single patient with extreme but valid test results is often very useful as part of the MA QC alarm protocol. All these aspects have recently been discussed in greater detail [10, 14].
Conclusions
Altogether, the recent developments in the field of PBRTQC and, more specifically, MA QC now – finally – enable true practical implementation of MA QC in medical laboratories and allow more effective and efficient QC plans to be designed.
The authors
Huub H. van Rossum1,2 PhD
1 Department of Laboratory Medicine, The Netherlands Cancer Institute, Amsterdam, The Netherlands
2 Huvaros, Amsterdam, The Netherlands
E-mail: h.v.rossum@nki.nl
