Journal of Computer-Assisted Linguistic Research

Computer-based Reading Recall on Sociolinguistic Research

Camila Franco Rodriguez — Tue, 12 Dec 2023 00:00:00 +0100

Global bilingual communities are a fascinating phenomenon that has received constant attention from different angles and disciplines. Sociolinguistic research has also turned interest towards what motivates change in these globalized settings, as well as psycholinguistic research has wanted to focus on the cognitive aspects of L2 speakers. With the widespread use of computer-based methods, it seems natural to add them to contemporary research as a way of understanding variation and change to a deeper level. Through the data I have collected, I debate in this article the importance of including computer-based tests as part of traditional variationist research. I argue that the traditional separation of methods and data collection has influenced the research process to a point where some new behaviors could be overlooked. In this article I report the relationship between cognitive adaptation and social experiences in the Colombian in the Philadelphia bilingual community, which becomes more proficient not only because of age and time of L2 learning, but also because of how welcoming their social circles are, as well as how diverse their friendships and workplaces are.

On Methods of Data Standardization of German Social Media Comments

Lidiia Melnyk, Linda Feld — Tue, 12 Dec 2023 00:00:00 +0100

This article is part of a larger project aiming at identifying discursive strategies in social media discourses revolving around the topic of gender diversity, for which roughly 350,000 comments were scraped from the comments sections below YouTube videos relating to the topic in question. This article focuses on different methods of standardizing social media data in order to enhance further processing. More specifically, the data are corrected in terms of casing, spelling, and punctuation. Different tools and models (LanguageTool, T5, seq2seq, GPT-2) were tested. The best outcome was achieved by the German GPT-2 model: It scored highest in all of the applied scores (ROUGE, GLEU, BLEU), making it the best model for the task of Grammatical Error Correction in German social media data.

A Lightweight Statistical Method for Terminology Extraction

Rogelio Nazar, Nicolás Acosta — Tue, 12 Dec 2023 00:00:00 +0100

We propose a method for the task of automatic terminology extraction in the context of a larger project devoted to the automation of part of the tasks involved in the production of terminological databases. Terminology extraction is the key to drafting the macrostructure of a terminological resource (i.e., the list of entries), to which information can be later added at the microstructural level with grammatical or semantic information. To this end, we developed a statistical method that is conceptually simple compared to modern neural network approaches. It is a lightweight method because it is based on term dispersion and co-occurrence statistics that can be computed with basic hardware. For the evaluation, we experimented with corpora of lexicography and linguistics in English and Spanish of ca. 66 million tokens. Results improve baselines in almost 20%.

Self-supervision of Hallucinations in Large Language Models: LLteaM

Sofía Correa Busquets, Lucas Maccarini Llorens — Tue, 12 Dec 2023 00:00:00 +0100

Large language models like GPT and Claude have revolutionized the tech industry over the past year. However, as generative artificial intelligence, they are prone to hallucinations. A large language model hallucinates when it generates false or nonsensical text. As these models improve, these hallucinations become less obvious and more dangerous for users. This research explores the phenomenon in the context of automated email response for customer service. First, it proposes a taxonomy of hallucinations in large language models based on their linguistic nature, and second, a multi-agent system that allows for the self-supervision of such hallucinations. This system generates email responses but prevents their delivery if hallucinations are detected, thus reducing the risks of generative AI in productive environments. Experiments with various state-of-the-art language models reveal that the only successful model’s operating costs currently exceed those viable for operational deployment. Moreover, a drastic performance drop after a recent update to GPT-3.5-turbo suggests likely shortcomings in industrial applications driven by retrieval-augmented generation. Overall, the research advocates for a Machine Linguistics to analyze the outputs of large language models, suggesting that such a collaboration between Linguistics and Artificial Intelligence could help mitigate the social risks of hallucination.