Scholarly Communication and Machine-Generated Text: Is it Finally AI vs AI in Plagiarism Detection?

Patit Paban Santra; Debasis Majhi

doi:10.17821/srels/2023/v60i3/171028

Authors

Patit Paban Santra
Assistant Librarian, DSMS College, Durgapur – 713206, Durgapur, West Bengal
Debasis Majhi
Research Scholar, Banaras Hindu University, Varanasi – 221002, Uttar Pradesh

DOI:

https://doi.org/10.17821/srels/2023/v60i3/171028

Keywords:

AI (Artificial Intelligence), GPT (Generative Pre-Training Transformer), Machine Learning, ChatGPT, Natural Language Processing (NLP), OpenAI, Plagiarism

Abstract

This study utilizes GPT (Generative Pre-Trained Transformer) language model-based AI writing tools to create a set of 80 academic writing samples based on the eight themes of the experiential sessions of the LTC 2023. These samples, each between 2000 and 2500 words long, are then analyzed using both conventional plagiarism detection tools and selected AI detection tools. The study finds that traditional syntactic similarity-based anti-plagiarism tools struggle to detect AI-generated text due to the differences in syntax and structure between machine-generated and human-written text. However, the researchers discovered that AI detector tools can be used to catch AI-generated content based on specific characteristics that are typical of machine-generated text. The paper concludes by posing the question of whether we are entering an era in which AI detectors will be used to prevent AI-generated content from entering the scholarly communication process. This research sheds light on the challenges associated with AI-generated content in the academic research literature and offers a potential solution for detecting and preventing plagiarism in this context.

Downloads

Download data is not yet available.

Metrics

Metrics Loading ...

References

Birunda, S. S. and Devi, R. K. (2021). A review on word embedding techniques for text classification. In J. S. Raj, A. M. Iliyasu, R. Bestak, and Z. A. Baig (Eds.), Innovative Data Communication Technologies and Application, p. 267-281. https://doi.org/10.1007/978-981-15-9651-3_23

Boden, M. A. and Edmonds, E. A. (2009). What is genera-tive art? Digital Creativity, 20(1-2), 21-46. https://doi. org/10.1080/14626260902867915

Brown, T. B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., Agarwal, S., Herbert-Voss, A., Krueger, G., Henighan, T., Child, R., Ramesh, A., Ziegler, D. M., Wu, J., Winter, C., … Amodei, D. (2020). Language models are few-shot learners, Available at: https://arxiv.org/ abs/2005.14165.

Chan, A. (2023). GPT-3 and InstructGPT: Technological dystopianism, utopianism, and ‘Contextual’ perspectives in AI ethics and industry. AI and Ethics, 3(1), 53-64. https://doi.org/10.1007/s43681-022-00148-6

Chowdhury, H. A. and Bhattacharyya, D. K. (2018). Plagiarism: Taxonomy, tools and detection techniques.

Cortiz, D. (2022). Exploring transformers models for emo-tion recognition: A comparision of BERT, DistilBERT, RoBERTa, XLNET and ELECTRA. Proceedings of the 2022 3rd International Conference on Control, Robotics and Intelligent System, 230-234. https://doi. org/10.1145/3562007.3562051

Crothers, E., Japkowicz, N., and Viktor, H. (2023). Machine generated text: A comprehensive survey of threat models and detection methods. Available at: https://arxiv.org/ abs/2210.07321

Das, A., Mandal, N., Rath, D. S. and Das, S. (2022). Trendline of open access publication by Indian Institute of Technology (IITs) researchers in India. SRELS Journal of Information Management, 399-409. https://doi.org/10.17821/srels/2022/v59i6/168621

King, M. R. and chatGPT. (2023). A conversation on artificial intelligence, chatbots, and plagiarism in higher educa-tion. Cellular and Molecular Bioengineering, 16(1), 1-2. https://doi.org/10.1007/s12195-022-00754-8

Labbé, C. and Labbé, D. (2013). Duplicate and fake publi-cations in the scientific literature: How many SCIgen papers in computer science? Scientometrics, 94(1), 379-396. https://doi.org/10.1007/s11192-012-0781-y

Maity, D. and Dutta, B. (2022). Identifying the core and allied disciplines involved in the growth of virology: A linguistic analysis. SRELS Journal of Information Management, 363-371. https://doi.org/10.17821/srels/2022/v59i6/170750

Oberreuter, G. and Velásquez, J. D. (2013). Text mining applied to plagiarism detection: The use of words for detecting deviations in the writing style-ScienceDirect. Expert Systems with Applications, 40(9), 3756-3763. https://doi.org/10.1016/j.eswa.2012.12.082

O’Connor, S. and ChatGPT. (2023). Open artificial intelli-gence platforms in nursing education: Tools for academic progress or abuse? Nurse Education in Practice, 66. https://doi.org/10.1016/j.nepr.2022.103537

Oladokun, B. D., Seidu, A. E., Ogunbiyi, J. O., Aboyade, W. A., Yemi-Peters, O. E. and Elai, M. A. (2022). Utilization of Information and Communication Technologies (ICTs) for managing students’ academic records in Nigerian Schools. SRELS Journal of Information Management, 373-381. https://doi.org/10.17821/srels/2022/v59i6/168449

Oya, M. (2020). Syntactic similarity of the sentences in a multi-lingual parallel corpus based on the Euclidean dis-tance of their dependency trees. Proceedings of the 34th Pacific Asia Conference on Language, Information, and Computation, 225-233.

Pal, A. and Mukhopadhyay, P. (2022). Fetching automatic authority data in ILS from Wikidata via OpenRefine. SRELS Journal of Information Management, 353-362. https://doi.org/10.17821/srels/2022/v59i6/170677

Parmar, R. D. and Nagi, P. K. (2022). Institutional knowl-edge repositories: Re-contextualization for accreditation and quality management. SRELS Journal of Information Management, 383-390. https://doi.org/10.17821/srels/2022/v59i6/170796

Pataranutaporn, P., Danry, V., Leong, J., Punpongsanon, P., Novy, D., Maes, P. and Sra, M. (2021). AI-generated characters for supporting personalized learning and well-being. Nature Machine Intelligence, 3(12). https:// doi.org/10.1038/s42256-021-00417-9

Petroni, F., Rocktäschel, T., Riedel, S., Lewis, P., Bakhtin, A., Wu, Y. and Miller, A. (2019). Language models as knowledge bases? Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), 2463-2473. https://doi.org/10.18653/v1/D19-1250

Roy, B. K., and Mukhopadhyay, P. (2022). Digital access brokers: Clustering and comparison (Part II - from Summarization to Citation Map). SRELS Journal of Information Management, 337-351. https://doi.org/10.17821/srels/2022/v59i6/170786

Topal, M. O., Bas, A. and van Heerden, I. (2021). Exploring transformers in natural language generation: GPT, BERT, and XLNet. Available at: https://arxiv.org/ abs/2102.08036

Transformer, G. G. P., Thunström, A. O. and Steingrimsson, S. (2022). Can GPT-3 write an academic paper on itself, with minimal human input?

van Noorden, R. (2014). Publishers withdraw more than 120 gibberish papers. Nature. https://doi.org/10.1038/nature.2014.14763

Wani, Z. A. and Bhat, A. (2022). Figshare: A one-stop shop for research data management with diverse features and services. SRELS Journal of Information Management, 391-397. https://doi.org/10.17821/srels/2022/ v59i6/170789

Weizenbaum, J. (1966). ELIZA-a computer program for the study of natural language communication between man and machine. Communications of the ACM, 36-45. https://doi.org/10.1145/365153.365168

Writer, B. (2019). Lithium-ion batteries: A machine-gener-ated summary of current research. Springer International Publishing. https://doi.org/10.1007/978-3-030-16800-1