Impact Evaluation Provider Directory

The Directory recognises individuals who have demonstrated training, experience, and publications in either of the following two areas, as defined by the Student Equity in Higher Education Evaluation Framework (SEHEEF) Guidance Manual:

• Quantitative Impact Evaluations (QIEs): QIEs aim to generate robust estimates of a program’s impact on target beneficiaries. This is achieved by comparing outcomes for the group receiving the intervention with a counterfactual control group that did not receive the intervention.
• Theory-Based Impact Evaluations (TBIEs): TBIEs focus on mapping the causal pathway from a program’s inputs to outcomes and collecting robust evidence to explain these links. To isolate the contribution of a specific program, the program’s logic is tested against other plausible explanations. As noted by HM Treasury (2020, p. 43): “For many of these (theory-based) methods, the aim is not to provide definitive evidence that the entirety of any measured change can be attributed to the intervention. Rather, they aim to explore whether the intervention definitively contributed to the measured change.”

Experience in other areas of evaluation or research, while valuable, is not considered relevant for this Directory.

All applicants must provide at least one publicly available publication that demonstrates the impact evaluation approach relevant to the stream for which they are applying.

The Directory is intended for individuals working in higher education equity within Australia. Applicants must be able to work competently and be able to work in the Australian context with appropriate work rights. This includes New Zealand citizens and permanent residents.

The Directory focuses on identifying individuals with demonstrated experience in conducting impact evaluations, providing universities with access to this specific expertise. Other relevant skills or attributes, such as lived experience related to disability, indigeneity, or socioeconomic disadvantage, are not assessed through the Directory. Institutions procuring evaluation specialists are responsible for identifying and considering these additional skills when selecting evaluators for specific projects.

For applicants to the Quantitative Impact Evaluation (QIE) stream, eligible designs focus on establishing an equivalent comparison group as the counterfactual. These may include experimental approaches (e.g. randomised controlled trials, randomised field experiments) and quasi-experimental designs (e.g. Propensity Score Matching, Regression Discontinuity Design, Difference-in-Differences, Interrupted Time Series, Synthetic Control Method, Instrumental Variables) with equivalent comparison groups.

For applicants to the Theory-Based Impact Evaluation (TBIE) stream, eligible designs focus on causal mechanisms and pathways within a theory of change (e.g. Realist Evaluation, Contribution Analysis, Process Tracing, Qualitative Comparative Analysis).

These specific types of Impact Evaluation designs should be clear in the application, in the “IE Experience” section. Other types of reviews, research or evaluation are not eligible “IE designs” for this specific panel.