Achievement of Learning Outcomes
Interim Stream Review · Prototype

Achievement of Learning Outcomes

What does the available evidence tell us about how well students are achieving the intended learning outcomes of this stream?

This workspace brings together what students demonstrate in program assessments, external licensure performance, and graduating-student self-report. These perspectives remain distinct so reviewers can understand what each contributes before considering the overall achievement picture.

Evidence at a Glance

What evidence am I reviewing?

Program performance

What students demonstrated through program assessments across Years 1–3 and academic years.

External performance

PEBC MCQ and OSCE results provide an external perspective on achievement. The two components remain separate because they use different measures.

Graduating-student perspective

AFPC and CCAPP self-report evidence shows how graduating students describe preparedness and where they report developing knowledge and skills.

Performance in Therapeutics

What do program assessments show about student performance in this stream?

Reviewer orientation

What stands out?

Year one dropped sharply then partially recovered

Year one fell sharply in the academic year two academic years ago and has risen across the two most recent academic years, though still below its initial level.

This academic year moved students into higher bands

Compared with the previous academic year, each program year shows fewer students in the lowest band and more clustered in the upper mid-range band.

How has performance changed within each program year?Read from the most recent academic year to earlier academic years within each program year.
<6060-<7070-<8080-<9090-100
Year 1
2025-2026
2024-2025
2023-2024
Year 2
2025-2026
2024-2025
2023-2024
Year 3
2025-2026
2024-2025
2023-2024
Course performance detail
PHARM 243Year 12025-202613974.3%74.2%8.4%
PHARM 243Year 12024-202513472.6%72.6%7.8%
PHARM 243Year 12023-202413671.4%71.2%7.2%
PHARM 343Year 22025-202613472.0%72.7%9.4%
PHARM 343Year 22024-202513371.2%71.3%10.2%
PHARM 343Year 22023-202413973.6%74.6%9.9%
PHARM 344Year 22025-202613371.7%72.6%7.8%
PHARM 344Year 22024-202513371.0%71.5%8.3%
PHARM 344Year 22023-202413977.2%78.0%8.3%
PHARM 345Year 22025-202613377.6%77.8%6.5%
PHARM 345Year 22024-202513374.8%74.7%7.5%
PHARM 345Year 22023-202413977.1%77.0%8.2%
PHARM 346Year 22025-202613373.9%73.8%7.8%
PHARM 346Year 22024-202513369.6%69.6%8.4%
PHARM 346Year 22023-202413970.3%70.9%8.6%
PHARM 443Year 32025-202613078.2%78.5%8.0%
PHARM 443Year 32024-202513576.5%76.7%8.3%
PHARM 443Year 32023-202412576.3%76.1%8.6%
PHARM 444Year 32025-202613075.2%75.4%8.6%
PHARM 444Year 32024-202513574.6%75.6%9.8%
PHARM 444Year 32023-202412572.9%72.4%8.5%
PHARM 445Year 32025-202613075.6%76.4%7.5%
PHARM 445Year 32024-202513574.7%74.3%7.8%
PHARM 445Year 32023-202412572.4%71.8%8.1%
PHARM 446Year 32025-202613078.2%77.4%7.0%
PHARM 446Year 32024-202513580.8%81.0%6.1%
PHARM 446Year 32023-202412680.2%79.9%6.5%
PHARM 448Year 32025-202613069.3%69.0%7.1%
PHARM 448Year 32024-202513562.6%63.5%8.2%
PHARM 448Year 32023-202412668.8%68.8%6.2%

Calculation note. For each student, exam scores were averaged within each course. Those course averages were then averaged to summarize performance across the stream. Years 1, 2 and 3 are shown separately.

Exam Reliability and Item Statistics Evidence

How well are our assessments functioning as measures of student performance?

How consistently are our exams measuring student performance?

Each dot is one written exam in the selected academic year. Higher values mean the questions on that exam worked together more consistently.

Year 1Year 2Year 3
0.400.500.600.700.800.901.00

Exam reliability is not course-grade reliability. A final course grade based on several related assessments may be more stable than any one exam, but course-grade reliability is not estimated here.

Exam reliability varies across individual assessments

Individual exams show variability in consistency across years, with Year one tending toward a narrower, more consistently higher range while Year two and Year three include wider spreads and lower minima.

What contributes to exam consistency? One important part is how the individual questions work together. Questions that are very easy or very difficult for most students can provide less information about differences in performance, while questions that do not separate stronger from weaker overall exam performance may deserve closer review.

Are our exam questions giving us useful information about differences in student performance?

Each dot is one scored question from written exams in the selected academic year. Choose a program year to bring those questions forward while keeping the rest in view.

1317 questions reviewed · 3% very difficult · 10% very easy · 20% low discrimination · 5% negative discrimination
0.000.250.500.751.00-0.20.00.20.40.6
How well the question separates stronger and weaker overall exam performance
How often students answered correctly — fewer ← → more

Discrimination is the key review signal

Discrimination — whether higher-performing students were more likely to answer correctly — most directly flags items that do not track overall exam performance, with Year two and Year three showing relatively more low or negative cases.

One important limit: question functioning and exam consistency tell us how the assessment is behaving. They do not tell us whether the exam adequately covers the intended learning.

Graduating-Student Perspective

How prepared do graduating students say they are, and where do they believe knowledge and skills were developed?

Self-reported preparedness by AFPC professional role

Graduating students rated how prepared they felt across the statements that make up each professional role. Within each role, read down from the most recent academic year to see how the response distribution has changed over time.

AFPC professional roleAcademic yearOverall graduating-student rating
Care Provider2025–26
2024–25
2023–24
Scholar2025–26
2024–25
2023–24
Very unpreparedUnpreparedSomewhat unpreparedSomewhat preparedPreparedVery prepared

Each row pools the six-point ratings across the governed statements contributing to that professional role. Point to or focus a bar for the median and share rated Prepared or Very prepared.

Self-reported knowledge & skill development

The eight lowest-rated stream-relevant topics in the most recent graduating-student survey are shown as full response distributions. The mean is used only to determine which topics appear.

TopicGraduating-student rating
traditional, complementary, and alternative medicine
nutrition
drugs in pregnancy and lactation
urology
dermatology
hematology
conditions affecting head, eyes, ears, nose, throat (EENT)
rheumatology
Very underdevelopedUnderdevelopedSomewhat underdevelopedSomewhat developedDevelopedVery developed
CCAPP knowledge & skill detail
cardiology2026595
cardiology2025592.4
cardiology2024583
clinical pharmacokinetics2025438.5
clinical pharmacokinetics2024427
complementary and alternative medicines2025443
complementary and alternative medicines2024439
conditions affecting head, eyes, ears, nose, throat (EENT)2026568.8
dermatology2026567.5
dermatology2025569.6
dermatology2024552
drug of abuse and dependency2025569.6
drug of abuse and dependency2024568.4
drugs in pregnancy2025553.2
drugs in pregnancy2024439.4
drugs in pregnancy and lactation2026558.8
endocrine2025583.5
endocrine2024576
endocrinology2026591.1
gastroenterology2026578.5
gastrointestinal2025587.3
gastrointestinal2024576.8
geriatrics2025444.3
geriatrics2024448
hematology2026567.5
hematology2025553.8
hematology2024442
infectious disease2026587.5
infectious disease2025581
infectious disease2024559
nephrology2026586.2
neurology2026585
neurology2025586.1
neurology2024572
nutrition2026443.8
nutrition2025443
nutrition2024441
oncology2026580
oncology2025565.8
oncology2024560
pain2026591.2
pain2025584.8
pain2024575
patient self-care and management of minor ailments2026586.2
pediatrics2025426.6
pediatrics2024428
psychiatric and mental health2026591.2
psychiatry/mental health2025578.2
psychiatry/mental health2024559
pulmonary2025586.1
pulmonary2024573
renal2025575.9
renal2024576
respirology2026587.5
rheumatology2026576.2
rheumatology2025559
rheumatology20244.550
self care/non-prescription drug-use2025587.3
self care/non-prescription drug-use2024571.7
sexual and reproductive health2026593.8
substance use disorders, dependency, and addiction2026583.8
traditional, complementary, and alternative medicine2026430
urology2026560
women’s and men’s health2025586.1
women’s and men’s health2024576

These are graduating-student self-reports. They describe perceived preparedness and development, not demonstrated competence.

Licensure Examination Performance

What does PEBC evidence add to the picture of achievement?

PEBC MCQ longitudinal performance

Within each area, read from the most recent reporting year to earlier reporting years. MCQ average item-difficulty scale. Lower values indicate stronger relative performance on this PEBC measure. National values are shown only where they are already available in the governed PEBC evidence. To keep the stream view focused, the figure shows the highest-relevance competencies identified by the governed PEBC-to-AFPC-to-stream crosswalk.

UAlbertaNational
0.000.501.00

Patient Care

2024–25
0.76
2023–24
0.76
2022–23
0.76

Product Distribution

2024–25
0.70
2023–24
0.72
2022–23
0.70

Health Promotion

2024–25
0.74
2023–24
0.71
2022–23
0.74

Knowledge and Research Application

2024–25
0.70
2023–24
0.73
2022–23
0.73

Quality and Safety

2024–25
0.72
2023–24
0.71
2022–23
0.78

PEBC OSCE longitudinal performance

Within each area, read from the most recent reporting year to earlier reporting years. OSCE competency-rating scale shown on the axis. National values are shown only where they are already available in the governed PEBC evidence. To keep the stream view focused, the figure shows the highest-relevance competencies identified by the governed PEBC-to-AFPC-to-stream crosswalk.

UAlbertaNational
50.0067.5085.00

Ethical, Legal and Professional Responsibilities

2024–25
50.60
2023–24
67.40
2022–23
84.00

Patient Care

2024–25
72.10
2023–24
71.50
2022–23
74.80

Product Distribution

2024–25
67.00
2023–24
71.30
2022–23
67.10

Knowledge and Research Application

2024–25
70.30
2023–24
70.90
2022–23
74.10

Quality and Safety

2024–25Unavailable
2023–24Unavailable
2022–23
81.90

PEBC OSCE global ratings

Within each area, read from the most recent reporting year to earlier reporting years. 1–4 OSCE global-rating scale. Higher values indicate stronger ratings. National values are shown only where they are already available in the governed PEBC evidence.

UAlbertaNational
1.002.504.00

Communication

2024–25
3.60
2023–24
3.60
2022–23
3.70

Outcome

2024–25
2.80
2023–24
2.80
2022–23
3.00

Overall performance

2024–25
3.00
2023–24
3.00
2022–23
3.20

MCQ and OSCE retain their own original measures and scales. Stream relationships establish relevance, not causal attribution.

Where the Evidence Connects

What becomes clearer when these perspectives are considered together?

Across the evidence

What becomes clearer?

When the program performance data, assessment-quality notes, graduating-student AFPC ratings, and PEBC results are viewed together a clearer, practice-oriented picture emerges. Year-over-year exam averages for Years 1–3 rose modestly in 2025–2026, and 2026 graduates report stronger preparedness for the Care Provider role than for Scholar. PEBC’s single-year snapshot is mixed: OSCE competency scores sit at or above national levels for Patient Care and Knowledge, while MCQ competency measures tend to be at or slightly behind national figures in areas such as Quality and Safety, Product Distribution, and Health Promotion. Assessment-quality components are available across program years, with multiple test-level reliability estimates reported.

Keep in mind

What should stay distinct?

These sources use different measures, directions, and reporting periods and therefore must be kept separate in interpretation. PEBC MCQ uses a lower-is-better scale while OSCE uses higher-is-better; many test-level reliabilities vary, with some estimates relatively low; and the PEBC and graduate survey items are single-year snapshots. Small numerical differences across these sources should be interpreted cautiously, and the measures should not be pooled or used to make causal claims.

Questions for Review

What questions does the achievement evidence warrant further consideration?

  • Which program years and specific assessments contributed most to the modest overall upward movement and the pronounced Year‑2 gain, and do those assessments have the stronger reliability profiles noted in the assessment‑quality evidence?
  • Do students’ stable/higher Care‑Provider perceptions and modest Scholar increases map onto the topic‑level internal improvements (e.g., cardiology, infectious disease) and the external licensure divergences (MCQ stability vs OSCE declines such as ethical/legal/professional responsibilities), and which mismatches require further inquiry?
  • Given the recurring absence of blueprint evidence and persistent flagged items across cycles, which evidence‑building steps (blueprint documentation, prioritized item revision, and focused OSCE station review) should be completed first to enable more confident cross‑evidence interpretation?
About this Evidence

Where did this information come from, and how was it prepared?

What information is included

This workspace brings together several different perspectives on achievement for the selected pharmacy stream: performance in program assessments, information about how written exams are functioning, graduating-student self-report, and PEBC licensure results.

These sources answer different questions and are kept separate. Program assessments show performance within the curriculum, graduating-student surveys describe perceived preparedness and development, and PEBC provides an external perspective after the program.

How program performance and assessment quality are shown

Program-performance figures summarize student exam performance within each course and then across courses in the stream. Program years are shown separately, and the most recent academic year can be compared with earlier academic years using the same calculation.

Assessment-quality figures provide additional context about how individual written exams and exam questions are functioning. Exam reliability and item statistics can help identify results that deserve closer review, but they do not establish whether an exam adequately covers the intended curriculum.

Exam reliability describes an individual assessment, not the reliability of a final course grade that combines several assessments.

How graduating-student ratings are shown

AFPC professional-role preparedness and CCAPP knowledge-and-skill development use six-point response scales. The coloured bars show the full distribution of student responses rather than reducing the result to a single score. Hover or keyboard focus provides the median and the share of responses in the two most favourable categories.

The professional-role figure compares the same role across available survey years. The knowledge-and-skill figure shows the eight lowest-rated stream-relevant topics in the most recent survey; the mean is used only to select those topics, while the displayed evidence remains the full response distribution.

These are student self-reports. They complement demonstrated achievement evidence but are not treated as direct measures of competence.

How PEBC results are shown

PEBC results are displayed on their original assessment-specific scales. MCQ competency results, OSCE competency results, and the available 1–4 OSCE global ratings are shown separately. Within each figure, the most recent reporting year appears first so changes can be followed over time.

UAlberta results are shown alongside the national value when that comparison already exists in the governed PEBC evidence. Missing national values are left absent rather than reconstructed.

PEBC reporting years are not treated as equivalent to the academic-cycle selector, and relationships to a stream establish relevance rather than showing that the stream caused the result.

How the written summaries and review questions are created

The report first prepares the quantitative evidence and its source-specific limits. AI-assisted writing is then used to create concise educator-facing What stands out notes, cross-evidence orientation, and review questions from those prepared records.

AI-assisted writing does not calculate the figures, combine unlike scales, decide whether learning outcomes have been achieved, determine why a pattern occurred, or recommend curriculum changes. Those judgments remain with the educators reviewing the evidence.

The written summaries are prepared before the report is displayed. Opening or browsing this report does not send a new request to an AI service.

How to interpret differences and missing information

A visible difference between academic years, survey years, or PEBC reporting years does not by itself establish an educationally important improvement or decline. Each source should first be interpreted on its own scale and within its own reporting period.

When evidence is missing, not comparable, unavailable for a stream, or reported on a different scale, the workspace keeps that limitation visible rather than filling the gap or creating a combined achievement score.