A Monte Carlo evaluation of masked visual analysis in response‐guided versus fixed‐criteria multiple‐baseline designs

Ferron and team ran 10,000 pretend multiple-baseline graphs on a computer. They tested two rules for deciding when to add the next baseline: wait until the data path is flat (response-guided) or always wait a set number of sessions (fixed-criteria).

Raters who did not know which graphs had real treatment effects judged the graphs while blind to condition. The goal was to see which rule kept false alarms under 5 % and still caught true effects 80 % of the time.

Masked visual analysis held false positives under .05 in every setup. It hit the 80 % truth target only when three things lined up: response-guided phase extension, large immediate jumps, and at least five baseline points per tier.

Fixed-length baselines often missed real effects unless the jump was huge. Waiting for flat baseline data before moving on gave the best mix of safety and power.

Fahmie et al. (2013) also tinkered with single-case rules, but they tested which control condition to use in a functional analysis. Both papers show that tiny design choices—what you wait for, what you compare—sway conclusions.

Titlestad et al. (2019) asked if VB-MAPP domain scores are reliable enough for goal setting. Their answer was “look at total scores, not single domains.” Ferron gives the same caution: single visual cues can fool you; use a stricter rule set.

Eussen et al. (2016) validated a new motor scale. Ferron’s work is the flip side: before you trust any scale’s data, make sure the visual rules you use to judge change are themselves valid.

Next time you run a multiple baseline, wait until each tier’s baseline is flat before starting treatment. Track at least five data points per tier. This simple habit keeps false positives low and still lets you spot strong interventions quickly.