Refining english writing proficiency assessment and placement in developmental education using NLP tools and machine learning

Da Corte, Miguel; Baptista, Jorge

http://hdl.handle.net/10400.1/29074

Utilize este identificador para referenciar este registo.

Nome:	Descrição:	Tamanho:	Formato:
133515.pdf		494.69 KB	Adobe PDF	Ver/Abrir

Contacte-nos

Autores

Da Corte, Miguel

Baptista, Jorge

Resumo(s)

This study investigates the enhancement of English writing proficiency assessment and placement for Developmental Education (DevEd) within U.S. colleges using Natural Language Processing (NLP) and Machine Learning (ML). Existing automated placement tools, such as ACCUPLACER, often lack transparency and struggle to identify nuanced linguistic features necessary for accurate skill-level classification. By integrating human-annotated linguistic features, this study aims to contribute to equitable and transparent placement systems that better address students’ academic needs, reducing misplacements and their associated costs. For this study, a 300-essay corpus was compiled and manually annotated with a refined set of 11 DevEdspecific (DES) features, alongside 328 linguistic features automatically extracted from CTAP and 106 via COH-METRIX. Supervised ML algorithms were used to compare ACCUPLACER-generated classifications with human ratings, assessing classification accuracy and identifying predictive features. This analysis revealed gaps in ACCUPLACER’s classification capabilities. Experimental results showed that models incorporating DES features improved classification accuracy, with Na¨ıve Bayes (NB) and Support Vector Machine (SVM) achieving scores up to 80%. The refined features presented and methodology offer actionable insights for faculty and institutions, potentially contributing to more effective DevEd course placements and targeted instructional interventions.