This Article

Citations


Creative Commons License
Except where otherwise noted, this work is licensed under Creative Commons Attribution-NonCommercial 4.0 International License.

Detecting and Accommodating Outliers in Meta-Analysis for Evaluating Effect of Albendazole on Ascaris lumbricoides Infection


1 Department of Biostatistics, School of Paramedical Sciences, Shahid Beheshti University of Medical Sciences, Tehran, IR Iran
2 Department of Basic Sciences, School of Rehabilitation Sciences, Shahid Beheshti University of Medical Sciences, Tehran, IR Iran
3 Department of Lab Sciences and Proteomics Research Center, School of Paramedical Sciences, Shahid Beheshti University of Medical Sciences, Tehran, IR Iran
4 Department of Hematology, Tarbiat Modares University, Tehran, IR Iran
*Corresponding Author: Khadijeh Najafi Ghobadi, Department of Biostatistics, School of Paramedical Sciences, Shahid Beheshti University of Medical Sciences, Tehran, IR Iran. Tel/Fax: +98-2122707347, E-mail: kh.najafi420@yahoo.com.
Iranian Red Crescent Medical Journal. 2014 May; 16(5): e17648 , DOI: 10.5812/ircmj.17648
Article Type: Research Article; Received: Jan 22, 2014; Revised: Mar 20, 2014; Accepted: Apr 9, 2014; epub: May 5, 2014; ppub: May 5, 2014

Abstract


Background:: Meta-analysis is a statistical technique in which the results of two or more independent studies, with similar objectives, are mathematically combined in order to improve the reliability of the results. The outliers, which may exist even in random models, can affect the validity and strength of meta-analysis results.

Objectives:: The current study uses "random effects variance shift model" to evaluate and correct the outliers in performing a meta-analysis study of the effect of albendazole in treating patients with Ascaris lumbricoides infection.

Patients and Methods:: The study used data from 14 clinical trials; each article was composed of two groups, a treatment group and a placebo group. These articles compared the effect of single dose intakes of 400 mg albendazole in treating two groups of patients with Ascaris lumbricoides infection. The articles were published in a number of internationally indexed journals between 1983 to 2013. For both groups in each article, the total number of participants, the number of those with Ascaris lumbricoides infection, and the number of those recovered following the intake of albendazole were identified and recorded. The relative risk (RR) and variance were computed for each article individually. Then, using meta-analysis, the RR was computed for all the articles together. In order to detect outliers the "random effects variance shift model" and "likelihood ratio test" (LRT) were used. Adopting the bootstrap method, the accuracy rates for sampling distribution of the tests, which were used for multiple testing, were obtained and the relevant graphs were depicted. For data analysis, STATA and R software were used.

Results:: According to meta-analysis results, the estimate for RR was 2.91, with a 95% confidence interval of 2.6 to 3.25. According to the method used in this study, three articles (articles number 4, 7, and 12) were outliers and, as such, they were detected in the graphs.

Conclusions:: We can detect and accommodate outliers in meta-analysis by using random effects variance shift model and likelihood ratio test.

Keywords: Meta-Analysis; Albendazole; Outliers; Random Effects Variance Shift Model; Ascaris lumbricoides

1. Background


In health and medical sciences, the same topic may be investigated in numerous studies that may sometimes reveal contradictory results. One way of achieving a conclusive result is the formulation of a meta-analysis study, which mathematically combines and analyses the results of different studies to achieve a more reliable outcome. In this method, the results from a number of individual articles with similar objectives are combined and then are subjected to more conclusive statistical analyses (1). The main objective of meta-analysis is the application of a test with a statistical power greater than that of the individual studies. Nonetheless, it is essential to pay due attention to factors that may systematically affect the overall results when commenting on the validity and strength of the results in a meta-analytic computation. One of the factors that can negatively affect the validity of results in a meta-analysis computation is the presence of outlier, an element of a data set that distinctly stands out from the rest of the data. The most thorough method of identifying distant data in terms of outliers has been formulated by Hedges and Olkin (2). Numerous graphic methods have been introduced for the evaluation of unusual samples, but these methods can just be used for fix models (3, 4). However, bearing in mind that there are mixed models and random models as well, it is better to investigate outliers through these models too (5, 6). Gumedze and Jackson (7) introduced methods of detecting and accommodating outliers in a meta-analysis work by a random effects variance shift model. Therefore, the current study uses Gumedze and Jackson's model to measure the degree or size of outliers in a meta-analysis of the effect of albendazole on patients with Ascaris lumbricoides infection. Ascaris lumbricoides is one of the most soil-transmitted helminthes (STH) in the world. It is estimated that 4.5 billion individuals are at risk of STH infection (Ascaris lumbricoides, hookworms, and Trichuristrichiura) and as many as 1.2 billion individuals might be infected with Ascarislumbricoides, with Ascaris lumbricoides, close to 800 million with Trichuris trichiura, and more than 700 million with hookworms (8, 9). The majority of STH infected individuals are children and the infections are an important factor contributing to malnutrition in this age group (10).

2. Objectives


The current study used "random effects variance shift model" to evaluate and correct the outliers in performing a meta-analysis study of the effect of albendazole in treating patients with Ascaris lumbricoides infection.

3. Patients and Methods


The current study used data from 14 clinical trial articles, investigating the effect of albendazole in treating patients with Ascaris lumbricoides infection (11-24). The articles had already been published in internationally referenced journals from 1983 to 2013. The articles were first obtained through different sources like the internet, data banks, and internationally recognized journals with some special criteria indicated below and then were subjected to the relevant meta-analyses. We used the terms “albendazole” in combination with “trial” or “study”, “ascariasis”, and “Ascaris lumbricoides”. Bibliographies of identified articles were screened for additional relevant studies. The other criteria such as sample size, age, diagnostic method, and dosage were checked in selected articles. The patients under study had been matched in terms of age in the studies under meta-analysis. Besides, all the studies had used a similar definition for recovery, the same amount and frequency of albendazole (a daily single dose of 400 mg of oral medication), and a similar binary response variable for recovery versus nonrecovery. For each of the 14 articles, the total number of participants, the number of those infected with Ascaris lumbricoides as well as those recovered following the intake of albendazole (for each of the two groups), the effect size, and variance of the intervention were computed. Bearing in mind that each study was composed of both the albendazole and the placebo groups, the responses produced would follow a dichotomous variable. To compare the effect of albendazole on Ascaris lumbricoides, the cure rates were used to compare two groups under study. The priority index of the effect of albendazole as cure ratio in intervention group to the placebo group was considered the relative risk (RR). The effect size or RR is shown by Ɵ; then test statistic has to be defined for the significance of the effect size. Test statistics is defined by Q = Σ Wi (yi − Ɵˆ) 2 in which Wi = 1 / Vi and Ɵˆ = Σ (WiYi) / Σ (Wi). Under the null hypothesis, for all the effect sizes, which were similar or symmetrical, the distribution of Q statistics was chi-square with K-1 degree of freedom (25). To detect outliers in the data of current study, the random effect Variance Shift Outlier Model was used. As such, this method was used to detect and test the outliers. For meta-analysis, the STATA software was used and the R software was employed to administer this method (26). A brief account of the method is provided below.

3.1. Random Effect Variance Shift Outlier Model (RVSOM)

Following Gumedze and Jackson (7), basic model on standard random effects for meta-analysis is as follows:

y = µ1n + u + e (1)

Where y is a n-vector of estimated treatment effects for the n independent studies, µ is the unknown overall treatment effect, 1n, is a n-vector of ones, u is a n-vector of unknown random effects, u ~ N (0و τ2 In) where τ2 is the between-study variance, which is unknown, e represents residual errors with, e ~ N (0,R) where R = diag (δ 21, δ 22,…, δ 2n). The elements of R, the study variances, are regarded as known. The variance-covariance matrix of (1) can then be written as var(y) = V = τ2In + R with the variance of the i th study treatment effect given as var(yi) = τ2 + δ2i.

3.2. Extending the Random Effects Model to the RVSOM

According to Gumedze and Jackson (7), the random effects variance shift outlier model (RVSOM) for the ith study (which allows an inflated variance for the ith study) takes the form:

Y = µ1n + δjdj + u + e (2).

This adds an extra term δjdj to model (1), where dj is the jth unit vector of length n, i.e. with value 1 in the ith position and zero elsewhere, and is an unknown random coefficient with δj ~ N (0,ω2j) for ω2j≥ 0.

The subscript j indicates which study has an inflated variance. Model (2) has the form of a simple linear mixed model with δj as a random effect with variance ω2j. The variance-covariance matrix for the data under the RVSOM for the jth observation is:

var (y) = ω2j dj d΄j + V

An extension of model (2), which allows different inflated variances for more than one study, can be written as:

Y = µ1n + DI δI + u + e

Where I is a subset {1, 2, …, r} of studies considered to be outliers, D = [dj] is an n × r matrix containing entires of 0 and 1, where an entry of 1 in the ith row and jth column indicates that study i has the jth of r inflated variances, and δI is a r × 1 vector of unknown random effects. We referred to this model as an ‘extended RVSOM’ (7).

3.3 Administering the Random Effect Variance Shift Outlier Model

At First, we used forest plot diagram to detect outliers in our data, then we entered the outliers detected in forest plot in the RVSOM Model as the jth observation. Then the model was fitted to the data and the degree of ωˆ 2j for the jth was computed; the larger size of ωˆ 2j, the more likely for it to detect as an outlier. The likelihood ratio test (LRT) was used to measure the size or magnitude of ωˆ 2j.

The null hypothesis was H0 2j = 0 against the alternative hypothesis was HA(j)2j > 0 for a RVSOM for observation j. Stram and Lee (27, 28) showed that the asymptotic null distribution of the test statistic for testing this type of hypothesis was a mixture of two chi-squared distributions on zero and one degree of freedom. However, Gumedze and Jackson (7) showed that for the RVSOM conditions it cannot be met; hence, following Gumedze et al. (29), we had to use a parametric bootstrap procedure to obtain the distribution of our test statistic.

3.4. Empirical Distribution of the LRT Statistic and Multiple Testing

Under the null hypothesis, when there are no outliers in the data, empirical distribution of the likelihood ratio test statistics by a parametric bootstrap procedure is as follow:

Step 1. Fit the null model (1) to the data to obtain estimates μ ̂ and τˆ2.

Step 2. Generate a new data vector from model (1) and estimates μ ̂ and τˆ2.

Step 3. Compute the likelihood ratio test statistics LRT j, j = 1,..., n, by fitting the model (2) to the simulated data and compute and save the order statistics of the set LRT j for j = 1,..., n.

Step 4. Repeat steps 2 and 3 R times. This step generates an empirical distribution of size R for each order statistic.

Step 5. Calculate the 100 (1-α)th percentile for each order statistic for the required significance level α. The percentiles, using α = 0.05 and k = 1 for largest order statistic, k = 2 for second largest order statistic, are shown in the plots given in the results.

4. Results


The data used in this study were taken from 14 clinical trial articles that investigated the effect of albendazole on patients with Ascaris lumbricoides infection; yi indicated the relative risk in the ith article. As the forest plot diagram (Figure 1) indicated, articles 4, 7 and 12 were different from the rest of studies. The results of RVSOM model are shown in Figure 2 and Table 1.

Figure 1.
Forest Plot Diagram Used to Investigate the Effect of Albendazole on Patients With Ascaris lumbricoides Infection

Figure 2a shows the estimates ω2j form the jth RVSOM and the next two plots, Figures 2b and 2c, show the corresponding estimates of the between study variance and the treatment effect. The plot 2d shows the likelihood ratio statistics from which we see that observations 4, 7 and 12 are clearly detected as expected outliers; in particular, its LRT statistic is around three times the threshold for the first order statistic. All these figures refer to the fact that articles 4, 7 and 12 had served as outliers.

Figure 2.
Random Effect Variance Shift Outlier Model Statistics Plotted Against Study Number for the Effect of Albendazole on Patients With Ascaris lumbricoides Infection

Table 1 includes the information of the articles used for meta-analysis, containing the names of the authors, year of publication, and the number of patients used albendazole regardless of its effect.

Table 1.
Results From Clinical Trial Articles Investigating the Effect of Albendazole on Patients With Ascaris lumbricoides Infection (Al) a (11-24)

Table 2 shows that the inferences were very robust when these three outliers were down weighted using the extended RVSOM (which includes three ω2j, j = 4, 7, and 12 terms in a single model) and our findings greatly alleviated any concerns about the potential impact of outliers in these data. Once again, our proposed methodology performed well.

Table 2.
Estimated Parameters for Models Fitted to Investigate the Effect of Albendazole on Patients With Ascaris lumbricoides Infection a

5. Discussion


The results obtained from meta-analytic evaluation of 14 studies investigating the effect of albendazole on patients with Ascaris lumbricoides, detected articles 4, 7, and 12 as outliers. The articles under study were published in the internationally indexed journals during 1983 to 2013. The study used RVSOM to detect outliers. The results obtained from the tests and the relevant figures might indicate that the studies in rows 4, 7, and 12 served as outliers in this meta-analytic review and that the findings were largely reliable. Of course, with application of the method used in article 30, the articles 2, 10, and 14 were also detected as outliers. In this regard, in another research, a meta-analysis study was conducted to investigate the effect of albendazole on two groups of case and control patients afflicted with Trichuris trichiura. The results indicated a relative risk estimate of 2.06 with 95% confidence intervals of 2.4 to 1.76. Therefore, with such results in mind, it can be claimed that, the cure ratio of individuals infected with Trichuris trichiura using albendazole was two times more than those who did not use the drug (22-24, 29, 30). In the current study, however, the relative risk estimate was 2.91 with 95% confidence interval of 2.6 to 3.25, indicating that the ratio of individuals infected with Ascaris lumbricoides using albendazole was nearly three times more than those who did not take the drug. Therefore, it can be concluded that albendazole has been more effective in treating Ascaris lumbricoides than curing Trichuris trichiura.

Acknowledgments

This manuscript was the result of a master degree thesis in the Shahid Beheshti University of Medical Sciences. We appreciate the School of Paramedical Sciences head and the deputy manager of the research department.

Footnotes

Implication for health policy/practice/research/medical education: Outliers in meta-analysis can affect the validity and strength of results; hence, by using the method of this article, researchers can detect and accommodate outliers in meta-analysis.
Authors' Contributions: All authors participated equally.
Financial Disclosure: There is no conflict of interest.

References


  • 1. Whitehead A. Meta-analysis of controlled clinical trials. John Wiley & Sons; 2003. ISBN 470854197.
  • 2. Hedges LV, Olkin I, Statistiker M. Statistical methods for meta-analysis. New York: Academic Press; 1985.
  • 3. Iyengar S, Greenhouse JB. Sensitivity Analysis and Diagnostics. Handbook of Research Synthesis and Meta-Analysis. 2009. p. 417-33.
  • 4. Wang MC, Bushman BJ. Using the normal quantile plot to explore meta-analytic data sets. Psychol Methods. 1998;3(1):46. [DOI]
  • 5. Shadish WR, Haddock CK. Combining estimates of effect size. The handbook of research synthesis and meta-analysis. 2009. p. 257-77.
  • 6. Viechtbauer W. Bias and efficiency of meta-analytic variance estimators in the random-effects model. J Educ Behav Stat 2005;30(3):261-93. [DOI]
  • 7. Gumedze FN, Jackson D. A random effects variance shift model for detecting and accommodating outliers in meta-analysis. BMC Med Res Methodol. 2011;11:19. [DOI] [PubMed]
  • 8. Bethony J, Brooker S, Albonico M, Geiger SM, Loukas A, Diemert D, et al. Soil-transmitted helminth infections: ascariasis, trichuriasis, and hookworm. Lancet. 2006;367(9521):1521-32. [DOI] [PubMed]
  • 9. Lammie PJ, Fenwick A, Utzinger J. A blueprint for success: integration of neglected tropical disease control programmes. Trends Parasitol. 2006;22(7):313-21. [DOI] [PubMed]
  • 10. Stephenson LS, Latham MC, Ottesen EA. Malnutrition and parasitic helminth infections. Parasitology. 2000;121 Suppl:S23-38. [PubMed]
  • 11. Chien FL, Foon KLP, Hassan K. Efficacy of albendazole against the three common soil-transmitted helminthiases. Tropical Biomed. 1989;6:133-6.
  • 12. Oyediran A, Oyejide CO. In: Series AiHRSoMICaS, editor(s). 1982.
  • 13. El-Masry NA, Trabolsi B, Bassily S, Farid Z. Albendazole in the treatment of Ancylostoma duodenale and Ascaris lumbricoides infections. Trans R Soc Trop Med Hyg. 1983;77(2):160-1. [PubMed]
  • 14. Bwibo NO, Pamba HO. Double-blind comparative study of albendazole and placebo in the treatment of intestinal helminths. Albendazole in Helminthiasis: Royal Society of Medicine International Congress and Symposium Series. 1984.
  • 15. Ovedoff DL, editor(s). Summary of albendazole trials in South-East Asia. Albendazole in Helminthiasis: Royal Society of Medicine International Congress and Symposium Series; 1984; p. 103-12.
  • 16. Upatham ES, Viyanant V, Brockelman WY, Kurathong S, Lee P, Chindaphol U. Prevalence, incidence, intensity and associated morbidity of intestinal helminths in south Thailand. Int J Parasitol. 1989;19(2):217-28. [PubMed]
  • 17. Stephenson LS, Latham MC, Kinoti SN, Kurz KM, Brigham H. Improvements in physical fitness of Kenyan schoolboys infected with hookworm, Trichuris trichiura and Ascaris lumbricoides following a single dose of albendazole. Trans R Soc Trop Med Hyg. 1990;84(2):277-82. [PubMed]
  • 18. Sinniah B, Chew PI, Subramaniam K. A comparative trial of albendazole, mebendazole, pyrantel pamoate and oxantel pyrantel pamoate against soil-transmitted helminthiases in school children. Trop Biomed. 1990;7(2):129-34.
  • 19. Beach MJ, Streit TG, Addiss DG, Prospere R, Roberts JM, Lammie PJ. Assessment of combined ivermectin and albendazole for treatment of intestinal helminth and Wuchereria bancrofti infections in Haitian schoolchildren. Am J Trop Med Hyg. 1999;60(3):479-86. [PubMed]
  • 20. Olds GR, King C, Hewlett J, Olveda R, Wu G, Ouma J, et al. Double-blind placebo-controlled study of concurrent administration of albendazole and praziquantel in schoolchildren with schistosomiasis and geohelminths. J Infect Dis. 1999;179(4):996-1003. [DOI] [PubMed]
  • 21. Kirwan P, Asaolu SO, Molloy SF, Abiona TC, Jackson AL, Holland CV. Patterns of soil-transmitted helminth infection and impact of four-monthly albendazole treatments in preschool children from semi-urban communities in Nigeria: a double-blind placebo-controlled randomised trial. BMC Infect Dis. 2009;9:20. [DOI] [PubMed]
  • 22. Ndibazza J, Muhangi L, Akishule D, Kiggundu M, Ameke C, Oweka J, et al. Effects of deworming during pregnancy on maternal and perinatal outcomes in Entebbe, Uganda: a randomized controlled trial. Clin Infect Dis. 2010;50(4):531-40. [DOI] [PubMed]
  • 23. Speich B, Ame SM, Ali SM, Alles R, Hattendorf J, Utzinger J, et al. Efficacy and safety of nitazoxanide, albendazole, and nitazoxanide-albendazole against Trichuris trichiura infection: a randomized controlled trial. PLoS Negl Trop Dis. 2012;6(6):ee1685 [DOI] [PubMed]
  • 24. Wiria AE, Hamid F, Wammes LJ, Kaisar MM, May L, Prasetyani MA, et al. The effect of three-monthly albendazole treatment on malarial parasitemia and allergy: a household-based cluster-randomized, double-blind, placebo-controlled trial. PLoS One. 2013;8(3):ee57899 [DOI] [PubMed]
  • 25. Viechtbauer W, Cheung MWL. Outlier and influence diagnostics for meta‐analysis. Res Synth Methods. 2010;1(2):112-25. [DOI]
  • 26. Viechtbauer W. Conducting meta-analyses in R with the metafor package. J Stat Softw. 2010;36(3):1-48.
  • 27. Stram DO, Lee JW. Variance components testing in the longitudinal mixed effects model. Biometrics. 1994;50(4):1171-7. [PubMed]
  • 28. Gumedze FN, Welham SJ, Gogel BJ, Thompson R. A variance shift model for detection of outliers in the linear mixed model. Comput Stat & Data Analysis. 2010;54(9):2128-44. [DOI]
  • 29. Keiser J, Utzinger J. Efficacy of current drugs against soil-transmitted helminth infections: systematic review and meta-analysis. JAMA. 2008;299(16):1937-48. [DOI] [PubMed]
  • 30. Alavi Majd H, Najafi Ghobadi KH, Ahmadi N, Nazari A. Application of effective diagnostic methods for meta-analyzing the effect of Albendazole in the treatment of patients with Hookworm. J Ilam Univ Med Sci. 2013;21(5):141-9.

Table 1.

Results From Clinical Trial Articles Investigating the Effect of Albendazole on Patients With Ascaris lumbricoides Infection (Al) a (11-24)

TrialAuthorsYearAlbendazolePlacebo
A1+A1-A1+A1-
1Oyediran and Oyejide1983225222
2El-Masry et al.1983110364
3Bwibo and Pamba1984364531
4Ovedoff1984160012
5Chien et al.19893741229
6Upatham et al19897442748
7Stephenson et al.199070015
8Sinniah et al.1990515010
9Beach et al.19996112339
10Olds et al19991794092137
11Patrick P et al.20096325385
12J. Ndibazza et al.201093019
13Speich B et al.20129007
14Wiria AE et al.20131446564174
a A1+ refers to those recovered following the treatment and A.1- refers to those who did not recover following the treatment.

Table 2.

Estimated Parameters for Models Fitted to Investigate the Effect of Albendazole on Patients With Ascaris lumbricoides Infection a

- Model M 0 Model M 1
Parameter Estimation 95% CI Estimation 95% CI
μ 11.247376 (8.8,13.7) 10.29457445 (5.16, 15.43)
τ2 20.083981 - 85.23284239 -
ω4 2 - - 140.53558626 -
ω7 2 - - 302.27428826 -
ω12 2 - - 0.01746554 -
a Lumbricoides data: Overall treatment effect (µ), Variance shift estimates for jth the study (ω2j), and between-study variance (τ2). M 0: Random effects; model M 1: Extended RVSOM for study 4,7, and 12.

Figure 1.

Forest Plot Diagram Used to Investigate the Effect of Albendazole on Patients With Ascaris lumbricoides Infection

Figure 2.

Random Effect Variance Shift Outlier Model Statistics Plotted Against Study Number for the Effect of Albendazole on Patients With Ascaris lumbricoides Infection
(2a) Variance shift estimates.ω2j; (2b) random effect variance estimates τˆ2; (2c) estimates for treatment effect µˆ; (2d) likelihood ratio test statistics for a RVSOM for study j, plotted against study number.