Differentiating translational English from original English using large language modelsDifferentiating translational English from original English using large language models

Abstract: Translated language often carries subtle linguistic markers that set it apart from text originally written in the target language. This study investigates the potential of powerful AI language models to automatically identify these differences, specifically focusing on distinguishing translated texts from original texts. More specifically, this study utilized transformer-based large language models to differentiate translational English and original English. FLOB (Freiburg–LOB Corpus of British English) was selected as the original English corpus, and COCE (Corpus of Chinese-English) was selected as the translational English corpus. Two models were tested: Bidirectional Encoder Representations from Transformers (BERT) and Robustly Optimized BERT Pretraining Approach (RoBERTa). The factors affecting classification results are investigated through SHAP analysis and analysis of text types that have significantly different error rates. Results show that both models achieved excellent performance, with F1-scores of .864 for BERT and .998 for RoBERTa. The text types miscellaneous, general fiction, skills trade, and hobbies, and humor exhibit significantly higher error rates. Reportage, review, and science exhibit significantly lower error rates. Through SHAP analysis, we find that higher error rates may be attributed to simpler sentences in these three text types and the shared characteristics of translational texts, such as the tendencies for simplification and explicitation. Conversely, lower error rates were associated with text types that did not share these characteristics. In summary, transformer-based large language models show great potential for the automatic classification and analysis of translational sentences.

摘要： 与目标语中的原创文本相比，译文往往带有一些细微的语言标记。本研究探讨强大的人工智能语言模型能否自动识别这些差异，重点在于区分译文与原创文本。具体而言，本研究采用基于 Transformer 的大语言模型区分翻译英语与原创英语，以 FLOB（Freiburg–LOB 英国英语语料库）为原创英语语料库，以 COCE（Corpus of Chinese-English）为翻译英语语料库，测试了 BERT 和 RoBERTa 两个模型，并通过 SHAP 分析以及对错误率差异显著的文本类型的分析，考察影响分类结果的因素。结果显示，两个模型均表现出色，BERT 的 F1 值为 .864，RoBERTa 为 .998。杂类、通俗小说、技能/行业与爱好、幽默等文本类型的错误率显著偏高，而新闻报道、评论和科学类文本的错误率显著偏低。SHAP 分析表明，错误率偏高可能是因为这些文本类型的句子较简单，且与译文具有某些共同特征，如简化和显化倾向；错误率偏低则与不具备这些特征的文本类型相关。总之，基于 Transformer 的大语言模型在翻译句子的自动分类与分析方面展现出巨大潜力。