Publicação
Toward consistency in writing proficiency assessment: mitigating classification variability in developmental education
| datacite.subject.sdg | 04:Educação de Qualidade | |
| datacite.subject.sdg | 10:Reduzir as Desigualdades | |
| datacite.subject.sdg | 09:Indústria, Inovação e Infraestruturas | |
| dc.contributor.author | Da Corte, Miguel | |
| dc.contributor.author | Baptista, Jorge | |
| dc.date.accessioned | 2026-06-02T16:34:01Z | |
| dc.date.available | 2026-06-02T16:34:01Z | |
| dc.date.issued | 2025 | |
| dc.description.abstract | This study investigates the adequacy of Machine Learning (ML)-based systems, specifically ACCUPLACER, compared to human rater classifications within U.S. Developmental Education. A corpus of 100 essays was assessed by human raters using 6 linguistic descriptors, with each essay receiving a skill-level classification. These classifications were compared to those automatically generated by ACCUPLACER. Disagreements among raters were analyzed and resolved, producing a gold standard used as a benchmark for modeling ACCUPLACER’S classification task. A comparison of skill levels assigned by ACCUPLACER and humans revealed a “weak” Pearson correlation (ρ = 0.22), indicating a significant misplacement rate and raising important pedagogical and institutional concerns. Several ML algorithms were tested to replicate ACCUPLACER’S classification approach. Using the Chi-square (χ2) method to rank the most predictive linguistic descriptors, Na¨ıve Bayes achieved 81.1% accuracy with the top-four ranked features. These findings emphasize the importance of refining descriptors and incorporating human input into the training of automated ML systems. Additionally, the gold standard developed for the 6 linguistic descriptors and overall skill levels can be used to (i) assess and classify students’ English (L1) writing proficiency more holistically and equitably; (ii) support future ML modeling tasks; and (iii) enhance both student outcomes and higher education efficiency. | eng |
| dc.identifier.doi | 10.5220/0013353900003932 | |
| dc.identifier.isbn | 978-989-758-746-7 | |
| dc.identifier.uri | http://hdl.handle.net/10400.1/29075 | |
| dc.language.iso | eng | |
| dc.peerreviewed | yes | |
| dc.publisher | SCITEPRESS - Science and Technology Publications | |
| dc.relation | Instituto de Engenharia de Sistemas e Computadores, Investigação e Desenvolvimento em Lisboa | |
| dc.relation.ispartof | Proceedings of the 17th International Conference on Computer Supported Education | |
| dc.rights.uri | http://creativecommons.org/licenses/by-nc-nd/4.0/ | |
| dc.subject | Developmental Education (DevEd) | |
| dc.subject | Automatic writing assessment systems | |
| dc.subject | English (L1) writing proficiency assessment | |
| dc.subject | Natural language processing (NLP) | |
| dc.subject | Machine-learning (ML) models | |
| dc.title | Toward consistency in writing proficiency assessment: mitigating classification variability in developmental education | eng |
| dc.type | conference object | |
| dspace.entity.type | Publication | |
| oaire.awardNumber | UIDB/50021/2020 | |
| oaire.awardTitle | Instituto de Engenharia de Sistemas e Computadores, Investigação e Desenvolvimento em Lisboa | |
| oaire.awardURI | info:eu-repo/grantAgreement/FCT/6817 - DCRRNI ID/UIDB%2F50021%2F2020/PT | |
| oaire.citation.conferenceDate | 2025 | |
| oaire.citation.endPage | 150 | |
| oaire.citation.startPage | 139 | |
| oaire.citation.title | In Proceedings of the 17th International Conference on Computer Supported Education (CSEDU 2025) | |
| oaire.citation.volume | 2 | |
| oaire.fundingStream | 6817 - DCRRNI ID | |
| oaire.version | http://purl.org/coar/version/c_970fb48d4fbd8a85 | |
| person.familyName | Da Corte | |
| person.familyName | Baptista | |
| person.givenName | Miguel | |
| person.givenName | Jorge | |
| person.identifier.ciencia-id | 7010-5366-22C5 | |
| person.identifier.orcid | 0000-0001-8782-8377 | |
| person.identifier.orcid | 0000-0003-4603-4364 | |
| person.identifier.rid | H-7699-2013 | |
| person.identifier.scopus-author-id | 14035269500 | |
| project.funder.identifier | http://doi.org/10.13039/501100001871 | |
| project.funder.name | Fundação para a Ciência e a Tecnologia | |
| relation.isAuthorOfPublication | 4a524eae-b359-47fa-8978-028ac5ffb57e | |
| relation.isAuthorOfPublication | e817fa28-a005-40e2-9ba4-03fdaedd7df3 | |
| relation.isAuthorOfPublication.latestForDiscovery | 4a524eae-b359-47fa-8978-028ac5ffb57e | |
| relation.isProjectOfPublication | 0b14d63a-8f78-4e31-8a86-b72e1f07871f | |
| relation.isProjectOfPublication.latestForDiscovery | 0b14d63a-8f78-4e31-8a86-b72e1f07871f |