Effects of Treatment on Disruptive Behaviors: A Quantitative Synthesis of Single-Subject Researches Using the PEM Approach (Report‪)‬

Meta-analysis provides a quantitative method to reach a certain conclusion by integrating relevant studies on a theoretical issue. Because the data of successive measurements over time in single-case experimental designs usually violate the assumptions of parametric statistics, especially that of homogeneity and independence of residuals, it is not appropriate to adopt methods used in conventional meta-analysis for between-group research. Mastropieri and Scruggs (1985-86) took a nonparametric approach, percentage of non-overlapping data (PND), to calculate the effect size for intra-subject research. Ma (2006) discussed the advantages and drawbacks of the PND and proposed an alternative method, the percentage of data points exceeding the median (PEM), to improve the shortcomings of the PND. Using original authors' judgment, i,e, the judgment of the author(s) of each located study, on the effectiveness of treatment as a validity criterion, the PEM approach had demonstrated a higher Spearman correlation with original authors judgment than the PND approach did. And this result was confirmed by Gao and Ma (2006). Scruggs et al. (1986, p.262) suggested a criterion to evaluate the effectiveness of treatment according to the PND scores: (a) highly effective when the score is above .90, (b) moderately effective when the score is between .70 and .90, (c) mildly (or questionable) effective in cases with scores between .50 and .70, and lastly, (d) ineffective, when the score is below .50. However, during coding, it was difficult to differentiate between moderately and mildly or questionable in the visual judgment of the effectiveness of treatment based on the curve change in the baseline and treatment phases. Ma (2006) and Gao and Ma (2006) incorporated "mildly (or questionable) effective" into "not effective." Hence these categories of effectiveness were coded: (a) highly effective, which was coded as 2, with core higher than .90, (b) moderately effective (coded as 1) with score equal to or higher than .70 but lower than .90, and (c) questionable or not effective (coded as 0) with score lower than .70. In the present study the authors try to incorporate "mildly (or questionable) effective" into "moderately effective" to form three categories: (a) highly effective with score above .90 (coded as 2), (b) partially effective including moderately and mildly (or questionable) effective (coded as 1) with score equal or greater than .50 but less than .90, and (c) ineffective (coded as 0) score less than .50, and to examine whether the superiority of validity of PEM approach over PND approach still sustains.