Assessment of Students’ Argumentative Writing: A Rubric Development

Burhan Ozfidan; Connie Mitchell

doi:10.29333/ejecs/1064

Authors

Burhan Ozfidan Florida Gulf Coast University
Connie Mitchell Prince Sultan University

DOI:

https://doi.org/10.29333/ejecs/1064

Keywords:

argumentative writing, rubric development, learning outcomes, assessment

Abstract

A rubric is used for assessing student work and performance. It is a tool that works in various ways to develop student learning and has great possibilities. The study presented aims to investigate the rubric development of second language learners' argumentative writing. The study's significance is to explore how well the rubric assesses students' achievement of the skills needed to develop argumentative essays. This study will add to the literature more data regarding rubrics' effectiveness in providing constructive feedback to students. This research describes the results of the current study in relation to rubric feedback from undergraduate students and the faculty who teach them from a private university in Saudi Arabia. The use of the rubric would be to support instruction and student performance. The researchers have proposed a methodology to design, develop, and implement a rubric as a scoring guide for argumentative essays based upon the achievement of learning outcomes for this genre. The rubric was developed to evaluate the following criterion: organization, integrating academic sources, thesis statement, finding evidence/lack of evidence, writing refutation paragraph, writing counterclaims, content, academic tone, mechanic, and vocabulary. The researchers statistically found significant interrater reliability and convergent validity coefficients. The results are considered to encourage the evaluation and development of such rubrics to be used across universities and colleges.

Downloads

Download data is not yet available.

References

Andrade, H. G. (2000). Using rubrics to promote thinking and learning. Educational Leadership, 57(5), 13–19.

Bangert-Drowns, R., Kulik C., Kulik, J., & Morgan, M. (1991). The instructional effect of feedback in test-like events. Review of Educational Research, 61, 213–238.

Birky, B. (2012). A good solution for assessment. Strategies: A Journal for Physical and Sport Educators, 25, 19–21.

Cabigao, J. (2021). Development and validation of a proposed model rubric in rating written outputs at the graduate school level. Development, 5(4), 122–131.

Carlson, K. D., & Herdman, A. O. (2012). Understanding the impact of convergent validity on research results. Organizational Research Methods, 15(1), 17–32.

Chase, C. (1999). Contemporary assessment for educators. Longman.

Chowdhury, F. (2019). Application of rubrics in the classroom: A vital tool for improvement in assessment, feedback and learning. International Education Studies, 12(1), 61–68. https://doi.org/10.5539/ies.v12n1p61

Crawford, A. R., Johnson, E. S., Zheng, Y. Z., & Moylan, L. A. (2020). Developing an understanding procedures observation rubric for mathematics intervention teachers. School Science and Mathematics, 120(3), 153–164.

Croux, C., & Dehon, C. (2010). Influence functions of the Spearman and Kendall correlation measures. Statistical methods & Applications, 19(4), 497–515.

Curran, V., Hollett, A., Casimiro, L. M., Mccarthy, P., Banfield, V., Hall, P., Lackie, K., Oandasan, I., Simmons, B, & Wagner, S. (2011). Development and validation of the interprofessional collaborator assessment rubric (ICAR). Journal of Interprofessional Care, 25(5), 339–344. https://doi.org/10.3109/13561820.2011.589542

Dawson, P. (2017). Assessment rubrics: Towards clearer and more replicable design, research and practice. Assessment & Evaluation in Higher Education, 42(3), 347–360. https://doi.org/10.1080/02602938.2015.1111294

Diab, R. & Balaa, L. (2011). Developing Detailed Rubrics for Assessing Critique Writing: Impact on EFL University Students' Performance and Attitudes. TESOL Journal, 2 (1), 52-72.

Ene, E. & Kosobucki, V. (2016). Rubrics and corrective feedback in ESL writing: A longitudinal case study of an L2 writer, Assessing Writing, 30, 3-20. https://doi.org/10.1016/j.asw.2016.06.003.

González, E. F., Trejo, N. P., & Roux, R. (2017). Assessing EFL university students' writing: A study of score reliability. Revista Electrónica de Investigación Educativa, 19 (2), 91–103. https://doi.org/10.24320/redie.2017.19.2.928

Hack, C. (2015). Analytical rubrics in higher education: A repository of empirical data. British Journal of Educational Technology, 46 (5), 924-927.

Halonen, J. S., Bosack, T., Clay, S., McCarthy, M., Dunn, D. S., Hill, G. W., IV, McEntarffer, R., Mehrotra, C., Nesmith, R., Weaver, K. A., & Whitlock, K. (2003). A rubric for learning, teaching, and assessing scientific inquiry in psychology. Teaching of Psychology, 30 (3), 196–208. https://doi.org/10.1207/S15328023TOP3003_01

Hyland, K. (2012). Second Language Writing. Cambridge.

Jeong, H. (2015). Rubrics in the classrooms do teachers really follow them? Language Testing in Asia, 5, Article 6. https://doi.org/10.1186/s40468-015-0013-5

Jonsson, A., & Svingby, G. (2007). The use of scoring rubrics: Reliability, validity, and educational consequences. Educational Research Review, 2, 130–144.

Lawshe, C. H. (1975). A quantitative approach to content validity 1. Personnel Psychology, 28 (4), 563–575.

Lee, J. E., Recker, M., & Yuan, M. (2020). The validity and instructional value of a rubric for evaluating online course quality: An empirical study. Online Learning, 24 (1), 245–263.

Mace, M. K., & Pearl, D. (2021). Rubric development and validation for assessing comprehensive internationalization in higher education. Journal of Studies in International Education, 25(1), 51–65.

McCarthy, K. S., Magliano, J. P., Snyder, J. O., Kenney, E. A., Newton, N. N., Perret, C. A., Knezevic, M., Allen, L. K., & McNamara, D. S. (2021). Quantified qualitative analysis: Rubric development and inter-rater reliability as iterative design. In Proceedings of the 15th International Conference of the Learning Sciences-ICLS 2021. International Society of the Learning Sciences.

McHugh, M. L. (2012). Interrater reliability: The kappa statistic. Biochemia Medica, 22(3), 276–282. https://doi.org/10.11613/BM.2012.031

Moskal, B. M., & Leydens, J. A. (2000). Scoring rubric development: Validity and reliability. Practical Assessment, Research, and Evaluation, 7(7),10-23. https://doi.org/10.7275/q7rm-gg74

Nimehchisalem, V., Chye, D. Y. S., & Jaswant Singh, S. K. A. (2014). A Self-Assessment Checklist for Undergraduate Students' Argumentative Writing. Advances in Language and Literary Studies, 5(1), 65-80.

Nitko, A. J. (2001). Educational assessment of students (3rd ed). Merrill.

Ozfidan, B., & Mitchell, C. (2020). Detected difficulties in argumentative writing: The case of culturally and linguistically Saudi backgrounded students. Journal of Ethnic and Cultural Studies, 7(2), 15–29.

Panadero, E., & Jonsson, A. (2013). The use of scoring rubrics for formative assessment purposes revisited: A review. Educational Research Review, 9, 129–144.

Reddy, Y. M., & Andrade, H. (2010). A review of rubric use in higher education. Assessment & Evaluation in Higher Education, 35(4), 435–448.

Stellmack, M. A., Konheim-Kalkstein, Y. L., Manor, J. E., Massey, A. R., & Schmitz, J. A. P., (2009). An assessment of reliability and validity of a rubric for grading APA-style introductions. Teaching of Psychology, 36(2), 102–107.

Tavakol, M., & Dennick, R. (2011). Making sense of Cronbach's alpha. International Journal of Medical Education, 2, 53–55. https://doi.org/10.5116/ijme.4dfb.8dfd

Trace, J., Meier, V., & Janssen, G. (2016). "I can see that": Developing shared rubric category interpretations through score negotiation. Assessing Writing, 30, 32–43. https://doi.org/10.1016/j.asw.2016.08.001

Tractenberg, R. E., Umans, J. G., & McCarter, R. J. (2010). A mastery rubric: Guiding curriculum design, admissions, and development of course objectives. Assessment & Evaluation in Higher Education, 35(1), 15–32.

Trainor, A. A., & Bal, A. (2014). Development and preliminary analysis of a rubric for culturally responsive research. The Journal of Special Education, 47(4), 203–216.

Turgut, F., & Kayaoğlu, M. N. (2015). Using rubrics as an instructional tool in EFL writing courses. Journal of Language and Linguistic Studies, 11(1), 47–58.

Wind, S. A. (2020). Do raters use rating scale categories consistently across analytic rubric domains in writing assessment? Assessing Writing, 43, 1–14. https://doi.org/10.1016/j.asw.2019.100416.

Wolf, K., & Stevens, E. (2007). The role of rubrics in advancing and assessing student learning. The Journal of Effective Teaching, 7(1), 3–14.

Wyngaard, S., & Gehrke, R. (1996). Responding to audience: Using rubrics to teach and assess writing. The English Journal, 85, 67–70.

Zhao, C. G. (2012). Measuring authorial voice strength in L2 argumentative writing: The development and validation of an analytic rubric. Language Testing, 30(2), 201–230. https://doi.org/10.1177/0265532212456965