Scientific articles

  • P. Przybyła, E. McGill, H. Saggion, “Attacking Misinformation Detection Using Adversarial Examples Generated by Language Models,” in Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing (EMNLP), Suzhou, China, 2025. [paper]
  • M. Shardlow, P. Przybyła, “Deanthropomorphising NLP: Can a language model be conscious?,” PLOS ONE, vol. 19, issue 12, 2024. [paper]
  • P. Przybyła, A. Shvets, H. Saggion, “Verifying the robustness of automatic credibility assessment,” Natural Language Processing, 2024.[paper]
  • I. Kuzmin, P. Przybyła, E. McGill, and H. Saggion, “TRIBBLE - TRanslating IBerian languages Based on Limited E-resources,” in Proceedings of the Ninth Conference on Machine Translation, Miami, USA, 2024.[paper]
  • A. Barrón-Cedeño, F. Alam, J. M. Struß, P. Nakov, T. Chakraborty, T. Elsayed, P. Przybyła, T. Caselli, G. Da San Martino, F. Haouari, M. Hasanain, C. Li, J. Piskorski, F. Ruggeri, X. Song, R. Suwaileh, “Overview of the CLEF-2024 CheckThat! Lab: Check-Worthiness, Subjectivity, Persuasion, Roles, Authorities, and Adversarial Robustness,” in Proceedings of the Fifteenth International Conference of the CLEF Association (CLEF 2024), Grenoble, France, 2024. [paper]
  • P. Przybyła, E. McGill, H. Saggion, “Know Thine Enemy: Adaptive Attacks on Misinformation Detection Using Reinforcement Learning,” in Proceedings of the 14th Workshop on Computational Approaches to Subjectivity, Sentiment, & Social Media Analysis, Bangkok, Thailand, 2024. [paper]
  • N. Duran-Silva, P. Accuosto, P. Przybyła, H. Saggion, “AffilGood: Building reliable institution name disambiguation tools to improve scientific literature analysis,” in Proceedings of the Fourth Workshop on Scholarly Document Processing (SDP 2024), Bangkok, Thailand, 2024. [paper]
  • P. Przybyła, B. Wu, A. Shvets, Y. Mu, K. C. Sheang, X. Song, H. Saggion, “Overview of the CLEF-2024 CheckThat! Lab Task 6 on Robustness of Credibility Assessment with Adversarial Examples (InCrediblAE),” in Working Notes of CLEF 2024 - Conference and Labs of the Evaluation Forum, Grenoble, France, 2024.[paper]
  • A. Barrón-Cedeño, F. Alam, T. Chakraborty, T. Elsayed, P. Nakov, P. Przybyła, J. M. Struß, F. Haouari, M. Hasanain, F. Ruggeri, X. Song, R. Suwaileh, “The CLEF-2024 CheckThat! Lab: Check-Worthiness, Subjectivity, Persuasion, Roles, Authorities, and Adversarial Robustness,” in Proceedings of the 46th European Conference on Information Retrieval (ECIR 2024), Glasgow, UK, 2024.[paper][preprint]
  • P. Przybyła, H. Saggion, “ERINIA: Evaluating the Robustness of Non-Credible Text Identification by Anticipating Adversarial Actions,” in Proceedings of the Workshop on NLP applied to Misinformation co-located with 39th International Conference of the Spanish Society for Natural Language Processing (SEPLN 2023), Jaén, Spain, 2023. [paper]
  • P. Przybyła, N. Duran-Silva, S. Egea-Gómez, “I've Seen Things You Machines Wouldn't Believe: Measuring Content Predictability to Identify Automatically-Generated Text,” in Proceedings of the 5th Workshop on Iberian Languages Evaluation Forum (IberLEF 2023), Jaén, Spain, 2023. [paper]

Reports

  • ERINIA Project Team, "The Robustness of Text Classification Models in Adversarial Applications", Universitat Pompeu Fabra, Barcelona, Spain, 2024. [paper]

Resources

BODEGA framework helps to evaluate the robustness of text classifiers, i.e. their ability to maintain the correct prediction for test examples that were modified by a malicious attacker. BODEGA is using tasks related to the detection of misinformation and aims to simulate the real usecase of social media platforms employing ML classifiers for content filtering. The framework is openly available on GitHub.

The repository includes the code for both the original evaluation procedure (described in the BODEGA article), as well as additional elements used in the InCrediblAE shared task. The models and data necessary to reproduce the results can be downloaded from the task repository.

Autext is a software package for building ML models performing identification of machine-generated text. The solution is based on measuring the predictability if the textual content as has received an award for the best performance in the AuTexTification shared task at IberLEF 2023. The source code is available for download from GitHub.

Xarello a solution for finding adversarial examples and thus testing the robustness of text classifiers, especially in the tasks of misinformation detection. Unlike other solutions, XARELLO is adaptive, i.e. it observes the responses from the victim classifier and learns, which attacks are succesfull in changing its decisions. The work was presented at the WASSA workshop and the source code is available from GitHub.

Trepat is another approach to finding adversarial examples in the tasks of misinformation detection. It focuses on the meaning preservation (by employing large language models in rephrasing) and realistic attack simulation (by imposing limits on the number of queries allowed). The work was presented at the EMNLP conference and the source code is available from GitHub.