Validating holistic scoring for writing assessment
In reflective interviews, the teachers also reported more consensus on their approach to evaluating grammar and organization than word choice and rhetoric.While these findings corroborate prior research comparing the rating of L1 and L2 writing, they promise to expand our understanding of rating processes by reflecting the teachers’ reading practices and attentional focus while rating.Raters with ESL training were chosen for this study because of their professional familiarity with features of L2 writing (Eckstein, Casper, Chan, & Blackwell, 2018) and a sensitivity to L2 writers’ needs.Results demonstrate specific areas of attentional focus which contribute to research of differences in the reading and rating of L1 and L2 writing.In his think-aloud study, Wolfe (2005) found evidence of raters’ cognitive behaviors on a variety of processing actions and content foci.Ultimately, he argued that less proficient raters tended to make and revise assessment decisions early in the rating task while more proficient raters withheld their assessments until after completing the text.
In a matched-guise study, Rubin and Williams-James (1997) applied Thai, Danish, and American guises to a stable set of six essays and collected holistic and analytic scores from 33 writing teachers.
Edgington (2005) found that teachers utilize numerous strategies, including “evaluating, clarifying, questioning, and inferring” (p. However, there remain questions of whether raters adopt the same reading behaviors in similar proportions to texts written by native English (L1) and non-native English (L2) writers.
The issue of reading for evaluation has gained relevance recently given that there are now more international students studying in U. higher education than ever before (Institute of International Education, 2016).
Researchers have found numerous differences in the approaches raters take to the complex task of essay rating including differences when rating native (L1) and non-native (L2) English writing.
Yet less is known about raters’ reading practices while scoring those essays.