10 things we learned from NAT 2023
This blog takes a deeper look at the National Achievement Test Findings Report 2023. We focus on the foundational learning aspects of the report, especially the results of the grade 4 level assessments, and draw out some key insights.
What is the National Achievement Test?
The National Achievement Test (NAT) is a government organised, sampled-based assessment that tests grade 4 and grade 8 students in English, Urdu, Sindhi, Maths, and Science. NAT 2023 took place across Pakistan in May 2023, with participation from over 23,000 students across nearly 1300 public schools. The last assessment was conducted in 2019.
What subjects were tested?
Grade 4 Maths, English, and Urdu/Sindhi were tested against specified grade 4 Student Learning Outcomes (SLOs) from the 2020 and 2022 National Curricula. For languages, the chosen SLO’s were:
Reading and Critical Thinking Skills and;
Formal and Lexical Aspects of Language.
For grade 4 Maths, the chosen content domains students were tested on were;
Numbers and Algebra
Measurement and Geometry, and;
Statistics and Probability.
What was the ‘foundational literacy indicator’?
Additionally, the NAT 2023 included a specific foundational literacy indicator which tested students on letter and word recognition. The purpose of the foundational literacy indicator was to assess basic literacy skills amongst students at grade 4, although it is worth noting that letter and word recognition is well below a Grade 4 competency and more closely related to Grade 2 competencies.
What are 10 things we learned about foundational learning from the NAT 2023?
Foundational literacy levels are high according to this measure (although letter recognition is pretty basic!). Grade 4 students achieved an impressive high of 88% on average in the foundational literacy assessment - an encouraging find. However, it is important to note that other reports, like the World Bank’s Learning Poverty indicator, indicate much lower foundational literacy levels in Pakistan.
English reading levels have improved, but from a low base. Students show improvement in test scores by 6-8% across all domains of Reading & Critical Thinking Skills they were tested on, compared to the NAT 2019. However, while scores in English reading improved by 5% in the test overall, they still only get about half (56%) of the questions right on the test. Hence, despite improvement, English reading still remains a low performing area.
Maths scores are low and display a concerning trend: students got less than half the answers right in grade 4, achieving an average score of 49%. The trend worsens in grade 8, where the average scores drop to 42%.
Urdu/Sindhi reading levels are good, with students achieving an average score of 68%.
Some students know a lot, others hardly know anything. The disparities in student performances are hard to miss. In the score distributions for grade 4 Maths and English, we see two peaks. The first peak occurs at the 25% score range (indicating that right answers were only achieved by guessing). The second peak is at a much higher performance level.
Girls outperform boys in all subjects except Maths, where both genders achieve similar average scores (this is in stark contrast to the ASER 2023 findings where girls performed worse throughout). Across provinces, the gender gap remains consistent, with no significant differences in scores observed. See the figure below.
Surprisingly, rural and urban students demonstrate similar performance levels! Provincially there is some variation: rural students in Balochistan score almost 6 points lower than urban students in grade 4 Maths, whereas in KP, rural students scored 4 points higher on the grade 4 English assessment than their urban peers.
Punjab stands out as the top performer amongst all the provinces: students achieved the highest scores of any province in grade 4 Maths, English, and Urdu/Sindhi, falling short of the top position in foundational literacy by only 0.4 points.
Islamabad Capital Territory performed very poorly: it ranks the lowest or second lowest in all grade 4 level indicators except foundational literacy, where it achieved the highest score.
Sindh performed the lowest out of all provinces on the foundational literacy indicator. Children scored an average of 10.8 out of 15 on the foundational literacy assessment. However, this could also be due to the test being translated into Sindhi, which might have impacted the difficulty level.
Rural Balochistan and urban Gilgit-Baltistan perform badly in grade 4 Maths. In both areas, the percentage of students achieving 25% or less marks, which indicates that correct answers were only achieved by guessing, is almost 40!
