Dale Taylor,Project Leader, ACS Doha International School
How do we measure the extemt to which we are cultivating well-rounded learners?
In spring 2013, ACS Doha International School concluded a year-long process aimed at articulating a school philosophy and a set of expected school-wide learning results (ESLRs). The ESLRs were developed in preparation for an accreditation visit by the Western Association of Schools and Colleges (WASC). Each WASC-accredited school is required to have a set of ESLRs, as well as providing evidence of student progress.
While the school has established methods for measuring students’ academic achievement, it had not considered how it would measure the extent to which students were ‘confident individuals, responsible global citizens or effective 21st Century contributors.’ While a growing number of schools around the world recognize the contribution of these non- cognitive student attributes to success in life, few had developed procedures for documenting them in the curriculum and assessing them.
While researching information about how other schools are thinking about assessing non-cognitive learner outcomes, the Educational Doha Admin Team (EDAT) came across the Mission Skills Assessment (MSA).1 The MSA originated in conversations between the US-based Independent Schools Data Exchange group (INDEX) and the Educational Testing Service’s (ETS) Center for Academic and Workforce Readiness and Success. ETS drew upon it extensive experience and designed a multi- method assessment incorporating and triangulating student self- assessment, situational judgment tasks and teacher ratings – the MSA.
The MSA is administered to students in grades six, seven and eight at the start of the second quarter of the school year. It is a 60-minute, computer- based assessment that focuses on six skills – creativity, resilience, curiosity, teamwork, ethics and time management.
Scores are reported by grade, gender and school – not individual performance, and can be correlated with academic grades, standardized test scores, attendance rates and other existing measures.
There are now more than 100 schools using the MSA, although ACS Doha was the first of only two schools outside the United States to join the project so far. Schools use the MSA results to examine specific subgroups (for example, differences between boys and girls), track cohort changes over time, and compare performance to other schools.
At the start of the 2014 – 2015 school year, the Early Childhood, Lower and Middle School principals at ACS Doha International School devised plans for introducing the six MSA skills in developmentally appropriate ways.
- Early Childhood teachers introduced the skills and then used MSA skills logo stickers to acknowledge when children were exhibiting the skills.
- Lower School teachers introduced the skills, used the same stickers to indicate where they were being addressed in their lesson planners, and encouraged students to address the skills in their reflective writing tasks.
- Middle School teachers introduced the six skills to students and parents and administered the test to all middle school students in November 2014.
Draft MSA results were received in April, with an opportunity to provide feedback to INDEX and ETS regarding the data presentation methods. Final results were received in June 2014.
What did we learn?
The following summaries present selected MSA results.
Differences by gender
Boys and girls self-assessments of the six skills were quite similar, while teachers consistently rated girls higher across all six skills. In most cases, girls and boys both assessed themselves lower than teachers’ ratings.
Differences by grade
Regardless of grade level, middle school students self-assessed themselves similarly across all six skills. In most cases, teachers’ ratings were higher than student self-assessment.
Differences by life satisfaction by gender and grade
The data suggests that as grade 6 students have a higher self-reported level of life satisfaction, with incremental drops noted from grade 6 to grade 7, and from grade 7 to grade 8. Overall, ACS Doha was below the INDEX average, but not in the bottom 10 schools.
School comparisons for six skills
- For CREATIVITY, ACS Doha was above the INDEX average, but not in the top 10 schools
- For CURIOSITY, ACS Doha was below the INDEX average, but not in the bottom 10 schools
- For ETHICS, ACS Doha was below the INDEX average, but not in the bottom 10 schools
- For RESILIENCE, ACS Doha was below the INDEX average, but not in the bottom 10 schools
- For TEAMWORK, ACS Doha was below the INDEX average, but not in the bottom 10 schools
- For TIME MANAGEMENT, ACS Doha was below the INDEX average, but not in the bottom 10 schools
About the testing process
The testing process was judged to be efficient, as careful planning had gone into laptop preparation and scheduling. As this is not a timed test, the testing period will be extended to 90 minutes in the future to allow EAL students additional time to read and respond.
The Early Childhood and Middle School divisions will continue with the approaches they initiated in September 2014.
The Middle School will develop teacher training materials and resources aimed at building expertise in teaching and assessing non-cognitive outcomes, including documenting these activities in the written curriculum. The Middle School will focus on two key areas, which the data suggests offer significant opportunities for improvement – resiliency and time management. The MSA again will be administered to middle school students in November 2015, as the first step in building a longitudinal picture of the effectiveness of the school’s efforts to cultivate these essential dispositions and skills.