The Power to Explain Variability in Intervention Effectiveness in Single-Case Research Using Hierarchical Linear Modeling
Use the free PowerSCED app to check you have enough participants before launching an HLM-based single-case study—20+ if you plan to test three moderators.
01Research in Context
What this study did
The team ran 10,000 fake studies on a computer. They wanted to know how many kids you need for a good HLM study.
They tested different numbers of kids, effect sizes, and moderators. They built a free web tool called PowerSCED.
What they found
You need at least the kids if you want to test three moderators. Fewer kids work only if the effect is huge.
The PowerSCED app gives you the exact number in seconds. Just plug in your planned study details.
How this fits with other research
MacNaul et al. (2021) reviewed 20 single-case studies on preference stability. Those studies used small samples. PowerSCED now shows why some may have missed real effects.
Cymbal et al. (2020) showed the PDC-HS works with real videos. Their sample was 30 staff. PowerSCED confirms that size is enough for one moderator, but tight for three.
Vassos et al. (2023) validated the 18-item SED-S with the adults. That sample passes PowerSCED's rule for two moderators. The new tool backs up their choice.
Why it matters
Before you start any HLM study, open PowerSCED. Type in your expected effect size and number of moderators. If the app says you need the kids and you only have 12, change the plan. This one click saves months of work that might fail to detect real effects.
Want CEUs on This Topic?
The ABA Clubhouse has 60+ free CEUs — live every Wednesday. Ethics, supervision & clinical topics.
Join Free →Go to PowerSCED.org and enter your next study's details to see if your sample is big enough
02At a glance
03Original abstract
This study investigated the power of two-level hierarchical linear modeling (HLM) to explain variability in intervention effectiveness between participants in context of single-case experimental design (SCED) research. HLM is a flexible technique that allows the inclusion of participant characteristics (e.g., age, gender, and disability types) as moderators, and as such supplements visual analysis findings. First, this study empirically investigated the power to estimate intervention and moderator effects using Monte Carlo simulation techniques. The results indicate that larger values for the true effects and the number of participants resulted in a higher power. The more moderators added to the model, the more participants needed to detect the effects with sufficient power (i.e., power ≥.80). When a model includes three moderators, at least 20 participants are required to capture the intervention effect and moderator effects with sufficient power. For that same condition, but only including one moderator, seven participants are sufficient. Specific recommendations for designing a SCED study with sufficient power to estimate intervention and moderator effects were provided. Second, this study introduced a newly developed user-friendly point and click Shiny tool, PowerSCED. This tool assists applied SCED researchers in designing a SCED study that has sufficient power to detect intervention and moderator effects. To end, the use of HLM with the inclusion of moderators was demonstrated using two previously published SCED studies in the journal School Psychology Quarterly.
Perspectives on Behavior Science, 2022 · doi:10.1007/s40614-021-00304-z