Brandon Muramatsu

Evaluability Assessment Method

While reading about Systematic Screening and Assessment, I got turned on to another concept, Evaluability Assessment. It’s another method that can be used to triage which programs should undergo full evaluation. (Note: I’ll use program and project interchangeably.)

Evaluability Assessment is described as, “A strategy that can be used to determine the extent to which a program is ready for full evaluation.” (Trevisan and Huang, 2003) And it has been described as a method to “assess the likelihood that prgram activities will reach mesuarable progress toward program objectives; and assessthe extent to which evaluation information is likely to be used by program management.” (Whorley, 1979)

I interpret this as the “reality check”–what I’m interested in in most projects is the project reasonable and is it likely to have a reasonable outcome.

Leviton and Gutman (2010) caution that, “EAs are a preevaluation activity, designed to maximize the chances that any subsequent evaluation will result in useful information.” (Which probably means I want to use them in a way for which it was not devised.)

I think the steps of the process that are most interesting to me, are the outcomes from Steps 6, 7 and 8.

  1. Describe Stakeholder Perceptions of Program.
  2. Identify Stakeholder Needs, Concerns, and Differences in Perceptions.  Differences in perception, needs, and concerns can indicate misperceptions of the program and intent, or a program that is not sufficiently meeting the needs of one or more stakeholder groups.
  3. Determine Plausibility of Program Model.  Data from program staff, documentation, and stakeholder interviews are used to determine plausibility of the program.  That is, data are analyzed to determine the extent to which the program is properly implemented, sufficiently developed, and activities appropriate, to reasonably predict that desired outcomes will be met.

References

Leviton, L. C., & Gutman, M. A. (2010). Overview and rationale for the Systematic Screening and Assessment Method. In L. C. Leviton, L. Kettel Khan, & N. Dawkins (Eds.), The Systematic Screening and Assessment Method: Finding innovations worth evaluating. New Directions for Evaluation, 125, 7–31.

Trevisan, M. S. & Huang, Y. M. (2003). Evaluability assessment: a primer. Practical Assessment, Research & Evaluation, 8(20). Retrieved August 8, 2010 from http://PAREonline.net/getvn.asp?v=8&n=20

Wholey, J. S. (1979).  Evaluation:  Promise and performance. Washington, DC: The Urban Institute.

/by