Browsing by Author "Baptista, Jorge"
Now showing 1 - 10 of 20
Results Per Page
Sort Options
- Assisting European Portuguese teaching: linguistic features extraction and automatic readability classifierPublication . Curto, Pedro; Mamede, Nuno; Baptista, JorgeThis paper describes two automatic systems: a linguistic features extractor and a text readability classifier for European Portuguese texts. Its main goal is to assist the selection of adequate reading materials to support Portuguese teaching, especially as a second language. To the feature extraction from texts, the system uses several Natural Language Processing (NLP) tools. Currently, 52 features are extracted: parts-of-speech (POS), syllables, words, chunks and phrases, averages and frequencies, among others. A classifier was created using these features and a corpus, previously annotated readability level, adopting the five-levels language classification official standard for Portuguese as Second Language. In a five-levels (from A1 to C1) scenario, the best-performing learning algorithm (LogitBoost) achieved an accuracy of 75.11% with a root mean square error (RMSE) of 0.269. In a three-levels (A, B and C) scenario, the best-performing learning algorithm (C4.5 grafted) achieved 81.44% accuracy, with a RMSE of 0.346.
- Authorship attribution in portuguese using character N-gramsPublication . Markov, Ilia; Baptista, Jorge; Pichardo-Lagunas, ObduliaFor the Authorship Attribution (AA) task, character n-grams are considered among the best predictive features. In the English language, it has also been shown that some types of character n-grams perform better than others. This paper tackles the AA task in Portuguese by examining the performance of different types of character n-grams, and various combinations of them. The paper also experiments with different feature representations and machine-learning algorithms. Moreover, the paper demonstrates that the performance of the character n-gram approach can be improved by fine-tuning the feature set and by appropriately selecting the length and type of character n-grams. This relatively simple and language-independent approach to the AA task outperforms both a bag-of-words baseline and other approaches, using the same corpus.
- Automated anonymization of text documentsPublication . Dias, Francisco; Mamede, Nuno; Baptista, JorgeSharing data in the form of text is important for a wide range of activities but it also raises a concern about privacy when sharing data that could be sensitive. Automated text anonymization is a solution for removing all the sensitive information from documents. However, this is a challenging task due to the unstructured form of textual data and the ambiguity of natural language. In this work, we present our implementation of an automated anonymization system, built in a modular structure, for documents written in Portuguese. Four different methods of anonymization are evaluated and compared. Two methods replace the sensitive information by artificial labels: suppression and tagging. The other two methods replace the information by textual expressions: random substitution and generalization. Evaluation showed that the use of the tagging and the generalization methods facilitates the reading of an anonymized text while preventing some semantic drifts caused by the remotion of the original information.
- Automatic generation of exercises on passive transformation in portuguesePublication . Baptista, Jorge; Lourenço, Sandra; Mamede, Nuno J.Technology plays a very important role in education and Intelligent Computer-Assisted Language Learning (iCALL) has emerged as a complementary or even alternative method to the conventional language teaching practices. The automatic generation (and correction) of language exercises based on real texts extracted from corpora constitutes a non-trivial challenge to iCALL tutorial systems, and may involve the use of sophisticated Natural Language Processing tools and large-scale linguistic resources. This paper presents the main issues related to the automatic generation of exercises on the Passive transformation, a commonly occurring type of exercises in language textbooks, but also a very complex topic of Portuguese grammar. The paper describes the methods used to produce a large batch of passive-active sentence pairs, where the active sentence was automatically generated from naturally occurring passive sentences, taken from a large-sized, publicly available, corpus. Sentence pairs are ranked by difficulty level. A sample of randomly selected sentence pairs (40 from difficult level, 100 from medium, and 100 from easy level) was manually evaluated by an expert. Results are presented and error analysis is performed. The sentence pairs can be used as prime and correct answer for iCALL systems.
- Avaliação de recursos computacionais para o portuguêsPublication . Gonçalves, Matilde; Coheur, Luisa; Baptista, Jorge; Mineiro, AnaTêm sido desenvolvidas várias ferramentas para o processamento da língua portuguesa. No entanto, devido a escolhas variadas na base dos comportamentos destas ferramentas (diferentes opções de pré-processamento, diferentes conjuntos de etiquetas morfossintáticas e de dependências, etc.), torna-se difícil ter uma ideia do desempenho comparativo de cada uma. Neste trabalho, avaliamos um conjunto de ferramentas gratuitas e publicamente disponíveis, que realizam as tarefas de Etiquetação Morfossintática e de Reconhecimento de Entidades Mencionadas, para a língua portuguesa. São tidos em conta doze modelos diferentes para a primeira tarefa e oito para a segunda. Todos os recursos usados nesta avaliaçãao (tabelas de mapeamento de etiquetas, corpora de referência, etc.) são disponibilizados, permitindo replicar/ afinar os resultados. Apresentamos ainda um estudo qualitativo de dois analisadores de dependências. Não temos conhecimento de nenhum trabalho similar recente, isto ´e, que tenha em conta as ferramentas atuais disponíveis, realizado para a língua portuguesa.
- Determinação de um mínimo paremiológico do português europeuPublication . Reis, Sónia; Baptista, JorgeO principal objetivo deste estudo é apresentar o ‘mínimo paremiológico’ do português europeu, isto é, a lista dos provérbios mais conhecidos e mais frequentemente utilizados pela generalidade dos falantes da comunidade linguística do português europeu. Para o estabelecimento do ‘mínimo paremiológico’ do português europeu foram utilizados diferentes procedimentos metodológicos ao longo de cinco anos. Numa primeira fase, coligiu-se uma base de dados digital com mais de 114.000 entradas (provérbios e variantes) e fez-se uma primeira seleção manual dos provérbios considerados mais usuais, tendo-se depois calculado a concordância entre anotadores, que foi bastante elevada. Procedeu-se também ao cálculo da frequência de provérbios e variantes (n. de ocorrências) em várias fontes: (1) em dicionários e coletâneas de provérbios; (2) num corpus de textos jornalísticos (CETEMPúblico, Santos & Rocha, 2001); (3) em manuais escolares de Português e de Português Língua não Materna; e (4) em dois motores de busca (Google e Bing). Procedeu-se ainda à aplicação de dois questionários distintos, online. Obteve-se assim uma lista de 318 provérbios – o mínimo paremiológico do português europeu. A lista dos 318 provérbios poderá́ ter diversas aplicações, quer para o desenvolvimento de instrumentos de diagnóstico ou terapia de certas patologias da linguagem, quer para a aprendizagem de português como língua estrangeira.
- Early experiments on automatic annotation of Portuguese medieval textsPublication . Bico, Maria Inês; Baptista, Jorge; Batista, Fernando; Cardeira, EsperançaThis paper presents the challenges and solutions adopted to the lemmatization and part-of-speech (PoS) tagging of a corpus of Old Portuguese texts (up to 1525), to pave the way to the implementation of an automatic annotation of these Medieval texts. A highly granular tagset, previously devised for Modern Portuguese, was adapted to this end. A large text (similar to 155 thousand words) was manually annotated for PoS and lemmata and used to train an initial PoS-tagger model. When applied to two other texts, the resulting model attained 91.2% precision with a textual variant of the same text, and 67.4% with a new, unseen text. A second model was then trained with the data provided by the previous three texts and applied to two other unseen texts. The new model achieved a precision of 77.3% and 82.4%, respectively.
- Estimating lexical availability of European Portuguese proverbsPublication . Reis, Sónia; Baptista, JorgeThis paper relates data on lexical availability with data on textual frequency of proverbs in European Portuguese. Each data source should provide different perspectives on the use of proverbs in the language. This should allow an empirically well-motivated selection of proverbs aiming at the development of NLP resources, specifically for applications for learning Portuguese as a Foreign Language and for the diagnosis/therapy of speech impairments/disabilities. A large database (over 114,000 proverbs and their variants) was independently classified by two annotators, according to intuitively estimated lexical availability. Next, a random, stratified sample was selected and lexical availability was then confirmed with an online survey. Frequency data was gathered from two web browsers and a large-sized, publicly available, corpus of journalistic texts. Results from the survey, the web and the corpus by and large confirm the initial intuitive classification and a core of commonly used proverbs was defined
- Um filtro para palavras exóticas frequentes em PortuguêsPublication . Baptista, Jorge; Faísca, LuísAs formas gráficas (tokens) que constituem as palavras de um texto são muitas vezes ambíguas, podendo frequentemente uma mesma forma corresponder a diferentes flexões de duas ou mais entradas lexicais distintas. Algumas dessas formas correspondem a palavras ‘exóticas’, isto é, palavras pouco frequentes ou até caídas em desuso. O objectivo deste estudo é a determinação, a partir do corpus do CETEMPúblico, das formas ambíguas mais frequentes de palavras exóticas do Português, com vista à construção de um filtro que, durante a fase de análise lexical, elimine o ‘ruído’ provocado por essas formas exóticas e que permita assim reduzir a ambiguidade formal dos textos, simplificando as fases posteriores do seu processamento automático.
- Let's play with proverbs? NLP tools and resources for iCALL applications around proverbs for PFLPublication . Reis, Sónia; Baptista, JorgeProverbs are an important form of cultural expression of a society and are related to various areas of knowledge and human experience (González Rey, 2002). While linguistic elements in widespread use, proverbs are very rich structures both from a cultural and from a linguistic point of view and can therefore contribute significantly to the teaching of languages, both native and foreign (Council of Europe, 2001). However, though there are extensive collections of Portuguese proverbs with tens of thousands of forms and its variants (Reis, in preparation), its automatic identification in texts is quite difficult, given its formal variation, both lexical and syntactic (Chacoto, 1994). Nevertheless, using real examples, where proverbs are used in a natural or spontaneous discourse context, is a more natural way to learn and teach the complex conditions and communicative situations that determine the use and meaning of these expressions. On the other hand, frequency indices associated with proverbs and its variants would allow one to select the most common expressions. These are precisely the most interesting forms from the point of view of their teaching/learning and could serve as a basis for the construction of educational games, particularly for learning Portuguese autonomously as a foreign language (PFL) assisted by computer. To make this possible, it is necessary, first of all, be able to recognize the occurrence of proverbs in the texts (Rassi et al. 2014), including the instances where these expressions are presented in a truncated or creatively modified form, for example, to better suit the communicative situation or to produce new and more expressive meanings. In this paper, we present an on-going project, which aims at automatic identification of proverbs in texts. In this interdisciplinary study, we combine natural language processing tools with questionnaires construction techniques for teaching purposes (Hoshino and Nakagawa 2005, Correia et al. 2010). This is illustrated here with different sets of formats that can be built based on the knowledge of the form and variation of proverbs, as well as their frequency in corpora.