Social validity assessments: is current practice state of the art?
Social validity checks should guide program tweaks, not just sit in a file.
01Research in Context
What this study did
The authors looked at how behavior analysts check if parents and teachers like our programs. They read lots of social validity forms and found most are just checkboxes. The paper gives six fixes to make these checks useful again.
What they found
Social validity surveys have turned into empty rituals. People fill them out but the data rarely changes the program. The paper says we need bigger consumer pools, better tools, and real client input.
How this fits with other research
Matson et al. (2009) also warn that many child social-skills tests have weak psychometrics. Both papers say pick tools with solid numbers behind them.
Gutierrez et al. (1998) show the Motivation Assessment Scale has poor reliability. Christopher et al. (1991) predict this problem when we use quickie surveys instead of sound measures.
Jia et al. (2019) give us hope. They trimmed three autism scales to short, psychometrically strong versions. It proves we can keep surveys brief and still valid.
Why it matters
Next time you plan a program, ask at least three stakeholder groups for input, not just parents. Swap your five-item smiley-face form for a tool with reliability data. Invite clients to help design goals and then rerun the survey mid-treatment so the numbers guide tweaks, not just sit in a file.
Want CEUs on This Topic?
The ABA Clubhouse has 60+ free CEUs — live every Wednesday. Ethics, supervision & clinical topics.
Join Free →Add one open-ended question to your current social validity form and pilot it with two teachers.
02At a glance
03Original abstract
The use of evaluative feedback from consumers to guide program planning and evaluation is often referred to as the assessment of social validity. Differing views of its role and value in applied behavior analysis have emerged, and increasingly stereotyped assessments of social validity are becoming commonplace. This paper argues that current applications of social validity assessments are straying from the point originally proposed for them. Thus, several suggestions for improving current social validity assessment are proposed, including (a) expanding the definition of consumers to acknowledge the variety of community members able and likely to affect a program's survival, (b) increasing the psychometric rigor of social validity assessments, (c) extending assessment to heretofore underrepresented populations, (d) implementing widespread application of well-designed social validity assessments, (e) increasing meaningful consumer involvement in the planning and evaluation of behavioral programs, and (f) educating consumers to make better informed programming decisions.
Journal of applied behavior analysis, 1991 · doi:10.1901/jaba.1991.24-189