Publicação
From prediction to precision: leveraging LLMs for equitable and data-driven writing placement in developmental education
| datacite.subject.sdg | 04:Educação de Qualidade | |
| datacite.subject.sdg | 10:Reduzir as Desigualdades | |
| datacite.subject.sdg | 09:Indústria, Inovação e Infraestruturas | |
| dc.contributor.author | Da Corte, Miguel | |
| dc.contributor.author | Baptista, Jorge | |
| dc.date.accessioned | 2026-04-09T09:40:00Z | |
| dc.date.available | 2026-04-09T09:40:00Z | |
| dc.date.issued | 2025 | |
| dc.description.abstract | Accurate text classification and placement remain challenges in U.S. higher education, with traditional automated systems like Accuplacer functioning as “black-box” models with limited assessment transparency. This study evaluates Large Language Models (LLMs) as complementary placement tools by comparing their classification performance against a human-rated gold standard and Accuplacer. A 450-essay corpus was classified using Claude, Gemini, GPT-3.5-turbo, and GPT-4o across four prompting strategies: Zero-shot, Few-shot, Enhanced, and Enhanced+ (definitions with examples). Two classification approaches were tested: (i) a 1-step, 3 class classification task, distinguishing DevEd Level 1, DevEd Level 2, and College-level texts in one single run; and (ii) a 2-step classification task, first separating College vs. Non-College texts before further classifying Non-College texts into DevEd sublevels. The results show that structured prompt refinement improves the precision of LLMs’ classification, with Claude Enhanced + achieving 62.22% precision (1 step) and Gemini Enhanced + reaching 69.33% (2 step), both surpassing Accuplacer (58.22%). Gemini and Claude also demonstrated strong correlation with human ratings, with Claude achieving the highest Pearson scores (ρ = 0.75; 1-step, ρ = 0.73; 2-step) vs. Accuplacer (ρ = 0.67). While LLMs show promise for DevEd placement, their precision remains a work in progress, highlighting the need for further refinement and safeguards to ensure ethical and equitable placement. | eng |
| dc.identifier.doi | 10.4230/OASIcs.SLATE.2025.1 | |
| dc.identifier.uri | http://hdl.handle.net/10400.1/28621 | |
| dc.language.iso | eng | |
| dc.peerreviewed | yes | |
| dc.publisher | Schloss Dagstuhl – Leibniz-Zentrum für Informatik | |
| dc.relation | Instituto de Engenharia de Sistemas e Computadores, Investigação e Desenvolvimento em Lisboa | |
| dc.rights.uri | http://creativecommons.org/licenses/by/4.0/ | |
| dc.subject | Large language models (LLMs) | |
| dc.subject | Developmental education (DevEd) | |
| dc.subject | Writing assessment | |
| dc.subject | Text classification | |
| dc.subject | English writing proficiency | |
| dc.title | From prediction to precision: leveraging LLMs for equitable and data-driven writing placement in developmental education | eng |
| dc.type | journal article | |
| dspace.entity.type | Publication | |
| oaire.awardNumber | UIDB/50021/2020 | |
| oaire.awardTitle | Instituto de Engenharia de Sistemas e Computadores, Investigação e Desenvolvimento em Lisboa | |
| oaire.awardURI | info:eu-repo/grantAgreement/FCT/6817 - DCRRNI ID/UIDB%2F50021%2F2020/PT | |
| oaire.citation.endPage | 18 | |
| oaire.citation.startPage | 1 | |
| oaire.citation.title | 14th Symposium on Languages, Applications and Technologies (SLATE 2025) | |
| oaire.fundingStream | 6817 - DCRRNI ID | |
| oaire.version | http://purl.org/coar/version/c_970fb48d4fbd8a85 | |
| person.familyName | Da Corte | |
| person.familyName | Baptista | |
| person.givenName | Miguel | |
| person.givenName | Jorge | |
| person.identifier.ciencia-id | 7010-5366-22C5 | |
| person.identifier.orcid | 0000-0001-8782-8377 | |
| person.identifier.orcid | 0000-0003-4603-4364 | |
| person.identifier.rid | H-7699-2013 | |
| person.identifier.scopus-author-id | 14035269500 | |
| project.funder.identifier | http://doi.org/10.13039/501100001871 | |
| project.funder.name | Fundação para a Ciência e a Tecnologia | |
| relation.isAuthorOfPublication | 4a524eae-b359-47fa-8978-028ac5ffb57e | |
| relation.isAuthorOfPublication | e817fa28-a005-40e2-9ba4-03fdaedd7df3 | |
| relation.isAuthorOfPublication.latestForDiscovery | 4a524eae-b359-47fa-8978-028ac5ffb57e | |
| relation.isProjectOfPublication | 0b14d63a-8f78-4e31-8a86-b72e1f07871f | |
| relation.isProjectOfPublication.latestForDiscovery | 0b14d63a-8f78-4e31-8a86-b72e1f07871f |