Achievement of Learning Outcomes

Calculation note. For each student, exam scores were averaged within each course. Those course averages were then averaged to summarize performance across the stream. Years 1, 2 and 3 are shown separately.

Calculation note. For each student, exam scores were averaged within each course. Those course averages were then averaged to summarize performance across the stream. Years 1, 2 and 3 are shown separately.

Calculation note. For each student, exam scores were averaged within each course. Those course averages were then averaged to summarize performance across the stream. Years 1, 2 and 3 are shown separately.

One important limit: question functioning and exam consistency tell us how the assessment is behaving. They do not tell us whether the exam adequately covers the intended learning.

One important limit: question functioning and exam consistency tell us how the assessment is behaving. They do not tell us whether the exam adequately covers the intended learning.

One important limit: question functioning and exam consistency tell us how the assessment is behaving. They do not tell us whether the exam adequately covers the intended learning.

These are graduating-student self-reports. They describe perceived preparedness and development, not demonstrated competence.

These are graduating-student self-reports. They describe perceived preparedness and development, not demonstrated competence.

These are graduating-student self-reports. They describe perceived preparedness and development, not demonstrated competence.

MCQ and OSCE retain their own original measures and scales. Stream relationships establish relevance, not causal attribution.

MCQ and OSCE retain their own original measures and scales. Stream relationships establish relevance, not causal attribution.

MCQ and OSCE retain their own original measures and scales. Stream relationships establish relevance, not causal attribution.

• 01What aspects of the Year‑3 performance profile (higher central tendency and tighter upper distribution) should reviewers examine to determine whether the improvement reflects meaningful curriculum learning gains or assessment/practice‑artifact effects?
• 02How should item‑functioning concerns (flagged items, negative discrimination, low alpha on some tests) change our confidence in fine‑grained interpretations of program scores, and which exams or items should be prioritized for psychometric review before using scores for curriculum decisions?
• 03Where do graduating students’ self‑reported strengths (Care‑Provider, Professional) and licensure signals (OSCE declines in some domains, low collaboration OSCE scores) diverge in ways that require further local investigation (for example, OSCE station design, rater training, or targeted curriculum in collaboration/leadership skills)?

• 01What explains the Year‑1 increase versus the Year‑3 decline in program performance, and to what extent do Year‑3 distributional concentration and exam‑level item issues account for that contrast?
• 02Does the OSCE drop in the Ethical/Legal/Professional domain align with graduates’ self‑reported Professional strengths, and what additional evidence (item/station review, rater data, or content mapping) is needed to reconcile the apparent divergence?
• 03How much do the documented absence of test‑blueprint/content‑validity documentation and the presence of flagged/negatively discriminating items limit attribution of program‑level score shifts to curriculum coverage or instructional change?

• 01To what extent does the Year‑3 rebound reflect a genuine improvement in candidate proficiency versus changes in item quality or exam composition, given the upper‑tail distributional shift and heterogeneous reliability across exams?
• 02Do graduates’ repeated lower self‑ratings for Scholar and Leader‑Manager roles (notably quantitative/management topics) correspond with PEBC signals (MCQ/OSCE domains) that suggest weaknesses in collaboration or applied quantitative skills, and what targeted curriculum or assessment evidence is needed to test that link?
• 03Given recurring item‑functioning concerns and absent blueprint/validity documentation, what immediate assessment‑quality actions (item review, removal, blueprint disclosure, rater‑training records) should be prioritized to increase confidence in using exam trends for curriculum decisions?