A study published in Discover Artificial Intelligence in November 2026 reveals that current Large Language Models (LLMs) struggle significantly with the translation of specialized English generative syntax terminology into Arabic. The research, conducted by Mohammed Q. Shormani and Yehia A. AlSohbani, specifically evaluated the performance of ChatGPT-5, the latest release from OpenAI, in handling 44 core technical terms from the field of generative syntax. The findings indicate that the model’s ability to accurately convey the syntactic properties embedded in these terms is limited, with a substantial portion of its translations being either inaccurate or only partially correct. This exploratory investigation highlights a persistent gap in the application of AI translation tools to highly specialized linguistic domains, suggesting that human expertise remains indispensable for maintaining terminological precision in academic and professional contexts.
The study was designed to assess the extent to which LLMs can capture the nuances of English syntax terminology when translating into Arabic. The researchers selected 44 terms from previous works in generative syntax, including books and journal articles, as well as from their own experience in the field. These terms reflect the historical development of generative syntax, from early stages to the Minimalist Program, and represent concepts typically taught at undergraduate levels. The dataset also included compound terms containing hyphens, such as C-command, to examine ChatGPT’s handling of structurally diverse terminology. The study intentionally narrowed its scope to provide an initial evaluation of ChatGPT’s handling of this specific domain, which has received very limited attention in existing literature.
To ensure a rigorous comparison, the researchers collected human translations of the 44 terms from academically reliable specialists. These human translations served as the benchmark for evaluating ChatGPT’s output. The study was conducted on September 28, 2025, using the free version of ChatGPT-5. The model was chosen over earlier versions, such as GPT-2, GPT-3, GPT-3.5, and GPT-4o, because it is the latest release and integrates several improvements over its predecessors. The researchers used a detailed prompt to instruct ChatGPT to translate the list of terms into Arabic, emphasizing the study’s objective to assess the model’s potential in identifying equivalent terms in Arabic syntax terminology.
The evaluation process involved a three-point human evaluation scale conducted by a referee panel of five specialists, including three in syntax, one in morphology, and one in translation studies. The panel categorized ChatGPT’s translations into three groups: correct, appropriate, and incorrect. Correct translations were defined as those that corresponded exactly, or with only insignificant syntactic or morphological variation, to the preferred Arabic equivalents identified by the expert panel. Appropriate translations were those that differed from the preferred equivalent but preserved the intended conceptual meaning and were considered semantically acceptable. Incorrect translations were those that failed to convey the intended technical meaning, introduced a substantially different concept, or employed a generic language equivalent incompatible with the term’s specialized meaning.
The quantitative analysis of the results revealed that only 11 out of the 44 terms, or 25%, were translated accurately by ChatGPT. These accurate translations matched the human translations in terms of syntax core interpretation. For example, the term Passive was correctly translated as المبني للمجهول, and Generative grammar was translated as النحو التوليدي. In contrast, 17 terms, or 38.6%, were classified as incorrect. These translations failed to convey the intended syntactic core properties or nuances. For instance, the term Government was rendered as الحكم, a political term unrelated to syntax, whereas the human translation was العامل, which is the conventional Arabic technical equivalent in generative syntax scholarship.
The remaining 16 terms, or 36.4%, were classified as appropriate. These translations contained some accuracy but deviated from the human rendering. For example, Minimalist Program was rendered by ChatGPT as البرنامج التبسيطي, where البرنامج is correct, but التبسيطي is misleading, as it suggests stylistic simplicity rather than the intended syntactic sense of الأدنوي. Another example is C-command, which ChatGPT translated as C أمر, a kind of half Englishization where C is English and أمر is Arabic, equivalent to command. However, the human translation was أمر المكوّن, which exactly manifests the intended meaning, where C is an abbreviation of Constituent. This suggests that ChatGPT may not have been adequately trained on such specialized terminology.
The study also identified specific translation strategies employed by ChatGPT, including translation by addition, morphological derivation, and Arabization and Englishization. Translation by addition refers to cases where ChatGPT translates a term by adding a phrase or explanation in parentheses or using “or.” For example, pro was translated as برو (ضمير مستتر غير ملفوظ يفترض وجوده), while the human translation was الضمير المستتر. Here, ChatGPT shows uncertainty by adding an explanatory phrase in parentheses, which partially approximates the human translation. In generative syntax, pro refers to a null subject pronoun in pro-drop languages such as Arabic and Hebrew, unlike English or French where null subjects are almost disallowed.
Morphological derivation differences also arose from how ChatGPT handles derivation compared to human translation. Human translators often derive adjectives or nouns beyond the given form. For instance, Sound-meaning Interface was translated by humans as وجيهة الصوت-دلالية, while ChatGPT translated it as واجهة الصوت والمعنى. Additionally, human translators derived مفاهيمي (adjective form of مفهوم), whereas ChatGPT produced التصوري, which is not accurate syntactically. A similar issue occurred with Linearization, which was rendered by ChatGPT as الخطية, while humans translated it as التخطيطية. The latter derives from the verb خطط ‘to mark lines,’ matching the intended meaning in generative syntax, where linearization refers to the process of arranging syntactic units into a linear order.
Arabization and Englishization were also observed in the translations. Arabization appears in cases like pro rendered as برو. Englishization appears in terms such as X-bar Theory rendered as نظرية X-bar by ChatGPT instead of the conventional نظرية س-, which has long been used in Arabic generative syntax. Although the data exists in ChatGPT’s training corpus, the model retained the English form, reflecting an alienating strategy. This may indicate a bias concerning Arabic language or that the term نظرية س is not “picked up” by ChatGPT. The study notes that there is a notable difference between Arabization/Englishization and transliteration, as the former involves adapting foreign terms to the phonological, morphological, and lexical systems of the target language, whereas transliteration primarily seeks to reproduce the source word’s sounds or letters as closely as possible.
The researchers argue that the high percentage of incorrect translations is due to a lack of domain-specific training data in the data ChatGPT has been trained on. This leads to the model’s mechanism in translating these terms resorting to polysemous terms that are not specific to the syntax domain. For example, ChatGPT translates the term Government as الحكم, because had it been trained on such terms, it would have been able to translate it as العامل, as manifested in human translation. The study emphasizes that translating English generative syntax terms into Arabic using ChatGPT still requires a hybrid workflow. Such a workflow between human and LLMs is vital to correct errors, address inadequacies, and prevent distortions of intended meaning, which could otherwise lead to confusion or ambiguity among students of generative syntax.
The study proposes actionable strategies to overcome these limitations and enhance ChatGPT’s translation performance. These strategies include finetuning LLMs on syntactic terminology in both languages to better the models’ working mechanism for accurate or at least appropriate translation. The researchers suggest the development of curated bilingual terminological databases of generative syntax, annotated corpora of Arabic linguistic literature, and standardized glossaries aligned with the generative enterprise. By compiling authoritative Arabic equivalents for core syntactic concepts and including this in LLMs’ training data, the models’ reliability, terminological consistency, and scholarly adequacy in specialized fields such as generative syntax could be significantly enhanced.
The study has several limitations. The sample of terms was limited to 44 generative syntax terms, which, though representing the most frequent terms, may not fully capture the diversity and complexity of specialized syntactic terminology in English and Arabic. The analysis relied primarily on a single AI model, ChatGPT-5, without comparing its performance to other AI translation tools or neural machine translation models. Additionally, the study focused exclusively on the translation of isolated terms without considering context-rich discourse or sentence-level use, which may influence translation accuracy and appropriateness. Future research could expand the dataset to include a wider range of syntactic terms and incorporate multiple AI translation models for comparative evaluation. This would allow for examining the performance of AI in translating full syntactic structures or contextually embedded examples to better assess its utility in longer contexts, perhaps within a discourse.
Subject of Research: Evaluation of ChatGPT-5’s accuracy in translating English generative syntax terminology into Arabic
Article Title: Current LLMs and English syntax terminology translation
Article References: Shormani, M. Q., & AlSohbani, Y. A. (2026). Current LLMs and English syntax terminology translation. Discover Artificial Intelligence, 6(1), Article 1833. https://doi.org/10.1007/s44163-026-01836-6
Image Credits: AI Generated
DOI: 10.1007/s44163-026-01836-6
Keywords: Large Language Models, Machine Translation, Generative Syntax, ChatGPT-5, Arabic Linguistics, AI Translation, Terminology Translation, Natural Language Processing, Current, LLMs, English, syntax

