Deanna Milne, Project Leader, ACS Doha International School
How can the assessment of non-cognitive skills improve skill development?
“Pause and reflection are often the casualties of our fast paced post- modern live. We have few opportunities to carve out space in our crowded schedules to reflect on who we are, why we do what we do, and who we should strive to become” (Bolin, 1999).
Reflection is one of the most important aspects of learning and can be argued to be one of the leading activities in contributing to the development of student’s non-cognitive skills. The importance of non- cognitive skills highlights the need for reflection in our schools, as well as the need to develop the skills necessary to create meaningful and evocative reflections that will lead to learning.
The International Baccalaureate Organisation also recognizes the importance of reflective practice, and in 2015 made changes to their IBDP Community Action Service (CAS) programme, shifting the focus of CAS from the number of hours of participation in CAS activities to the quality of student reflection. However, the IBDP stopped short of providing guidance for assessing the
quality of reflections.
This research project asked the question ‘How do we help students understand how to create meaningful reflections and how do teachers assess them?” The project’s aim was draft and test a rubric that would give students an overview of the skills required to create reflections, and how to use the rubric and rubric feedback to improve their reflective practice.
A literature review, Service Learning and Critical Thinking7, was written by project leader, Deanna Milne, in preparation for an action research project, undertaken as a requirement of a Concordia University Masters degree in Education programme. Her initial action research project gave rise to her current research question, and the literature review informed both action research cycles.
One Day Workshop
A project team consisting of AS Coordinators for the IBDP Program from three of the four ACS campuses was formed to take this project forward. The team met for a one-day workshop to brainstorm what effective reflection ‘looks’ like. Starting with the big ideas the team narrowed their ideas down to the three things a reflection must look at: the what; the so what; and the now what.
The indicators identified to signify an effective reflection fell into four categories: descriptive, evaluative, interpretive, and communicative. Communicative was added to the typical categories of self-reflection because although students have a myriad of ways to reflect, and can be as creative as they like, the reflection must still effectively communicate the process, learning and next steps.
At the end of the workshop the team had created a draft rubric for assessing reflections that measured the quality of the reflection based on the four categories, with specific indicators for each. Reflections are marked on a pre-emergent, emergent, meets expectations, and exceeds expectations continuum.
Testing the Rubric
The team recruited additional teachers to take part in testing the rubric. To do this, the extended project team engaged in a moderation activity, sharing past student reflections and assessing them using the draft rubric.
For the next stage of testing the teachers randomly sample 10% of their population (Grade 11 and 12 students), assessing two reflections that were done prior to the students receiving the rubric, and two reflections that were done after the students had received the rubric. The revised rubric was introduced to the students at all campuses in a mini-workshop where rubric use was explained and students an opportunity to seek clarification and ask questions.
Over the next three months, students completed reflections and the project team used the revised rubric to assess the student reflections. It was agreed by the project team that because this was a pilot trial of the rubric, marks would be recorded in a ‘best fit’ for each of the criteria to see how much improvement students made overall within that criteria.
What did we learn?
The data gathered suggested that the rubric served as an assistive tool for students while creating reflections.
On the whole the data showed student progress in understanding the practice of self-reflection. The data suggested that as students develop their reflection skills conceptual thinking is more challenging, and that cultural differences may also play a role in the practice and representation of reflection.
In a few cases, students showed no improvement or reflection quality declined. This suggests that while the rubric may provide explicit information about what to improve, rubric use must be supported by additional teaching strategies that address how to improve.
In interviewing some students about reflection in general, the comments ranged from ‘It really helps me because I can see how my learning is progressing,’ to, ‘having to write a reflection doesn’t really make sense to me – it seems really American.’
The team will be continuing with their research in 2015-2016 having proposed a related research question, ‘ How can a continuum (Lower School to High School) of reflective writing assessment and developmentally appropriate reflective writing tasks enhance student learning (defined as growth and developmental in knowledge and understanding, skills and dispositions) and meta-cognition?’
The new project will build on the experience of setting and assessing student reflective writing tasks for IB students, to develop a developmentally appropriate continuum for students reflective writing tasks and assessments in order to build the knowledge, skills and dispositions we want all our students to develop so that through learning, they are best prepared to make a difference.