Post-trial review report: Management of academically at-risk Bachelor of Commerce students at Curtin University

Executive summary

Brief overview of the trial

This study aimed to evaluate the impact of targeted academic support provided to academically at-risk students—or Academically At-Risk Student Management (AARSM) practice—through a parallel randomised controlled trial (RCT) in Semesters 1 and 2, 2024. This academic support included the identification of academically at-risk second- and third-year Bachelor of Commerce (B.Com) students, and varying levels of targeted academic support. Due to challenges with gaining access to data in Semester 1, 2024 (detailed in Section 5), this phase was treated as a feasibility study, with this report focusing exclusively on Semester 2, 2024 data.

In Semester 2, 2024, the trial encompassed a population of 1,800 B.Com students, with 901 identified as academically at-risk, forming the study sample. These students were allocated into four groups (approximately 225 per group) receiving different levels of intervention intensity. However, inconsistent availability of assessment result data and issues arising relating to the quality of available data compromised the validity of the trial.

Summary of outcomes

The trial revealed shortcomings in the reliability and timeliness of data, impeding the ability for the trial to be implemented as intended.

As such, the trial was unable to determine whether the AARSM initiative resulted in statistically significant improvements in student engagement with the Learning Management System (LMS). No statistically significant increases in pass rates were observed across the intervention intensity levels.

However, the control group’s fail rate (11%) was nearly double the faculty average (approximately 6%), suggesting the identification of students at elevated academic risk may be a useful indicator of later success. Consequently, the criteria employed for identifying at-risk students in this trial hold promise for future studies

Key lessons and recommendations

• To enhance data integrity and operational feasibility, future trials should incorporate comprehensive pre-trial scoping and a pilot, lead-in phase that focuses specifically on validating data accuracy and reliability, and ultimately suitability for evaluation purposes.
• Achieving an adequately powered trial necessitates suitable planning and consideration of appropriate contingency measures tailored to specific contextual requirements and challenges as they emerge. Continuous expert guidance is critical during both the design and implementation phases to ensure methodological rigor and effective execution.

The full report is available for download in PDF [358 KB] or Word [246 KB] format.