Machine Learning and Bibliographic Data Universe: Assessing Efficacy of Backend Algorithms in Annif through Retrieval Metrics


  • Department of Library and Information Science, University of Kalyani, Kalyani − 741235, West Bengal



Annif, Automated Indexing, Machine Learning, NDCG, Neural Network Model, Retrieval Metrics.


This research study utilizes an open source AI/ML framework named Annif, developed by the National Library of Finland, to explore the feasibility of automated subject indexing. The framework loads the linked open data format of LCSH and trains the model with a comprehensive training dataset comprising MARC records downloaded from different libraries all over the world. It then compares a set of selected machine learning backends of Annif, namely TF-IDF, Omikuji, and Neural Network, against a set of retrieval metrics to measure the suitability of these backends for the bibliographic data universe. The study concludes that the fusion backend in Annif named Neural Network has the potential to provide support for an automated subject indexing system.


Download data is not yet available.


Metrics Loading ...


Anderson, J. D., & Pérez-Carballo, J. (2001). The nature of indexing: How humans and machines analyze messages and texts for retrieval. Part I: Research, and the nature of human indexing. Information Processing & Management, 37(2), 231-254.

Asula, M., Makke, J., Freienthal, L., Kuulmets, H.-A., & Sirel, R. (2021). Kratt: Developing an automatic subject indexing tool for the national library of Estonia. Cataloging and Classification Quarterly, 59(8), 775-793.

Frank, E., & Paynter, G. W. (2004). Predicting Library of Congress classifications from Library of Congress subject headings. Journal of the American Society for Information Science and Technology, 55(3), 214-227.

Golub, K. (2021). Automated subject indexing: An overview. Cataloging and Classification Quarterly, 59(8), 702-719.

Hahn, J. (2021). Semi-automated methods for bibframe work entity description. Cataloging and Classification Quarterly, 59(8), 853-867. 9374.2021.2014011

Hahn, J. (2022). Cataloger acceptance and use of semiautomated subject recommendations for web scale linked data systems. IFLA WLIC, 2022, 10. Available from:

Joorabchi, A., & Mahdi, A. E. (2013). Classification of scientific publications according to library controlled vocabularies: A new concept matching-based approach. Library Hi Tech, 31(4).

Junger, U. (2017). Automation first - the subject cataloguing policy of the Deutsche Nationalbibliothek. Available from:

Möller, G., Carstensen, K.-U., Diekmann, B., & Wätjen, H. (1999). Automatic classification of the world-wide web using the universal decimal classification. National Agricultural Library. (2014). NFAIS Webinar: automated indexing: A case study from the national agricultural library | ISSN. newsletter_issn/nfais-webinar-automated-indexing-acase-study-from-the-national-agricultural-library/

National Library of Medicine (NLM). (2002). NLM Medical Text Indexer (MTI).

Mukhopadhyay, P. (2022). AI/ML applications for knowledge organization in libraries: Designing a semi-automated subject indexing system based on LCSH. In M. Visakaruban et al., (Eds.), Proceedings of the Etakam Research Conference: enhancing library system to best engage with Global Change 2022 (ERC 2022) (pp. 11-19). University of Jafna.

OCLC. (2022, June 8). Scorpion. OCLC. Available from:

Oliver, C. (2021). Leveraging KOS to extend our reach with automated processes. Cataloging and Classification Quarterly, 59(8), 868-874.

Silvester, J. P. (1997). Computer supported indexing: A history and evaluation of NASA’s MAI system. Supplement 24. Undefined. Available from: 26fd1bd245edaf26a67c5f696a65b5032

Suominen, O. (2019). Annif: DIY automated subject indexing using multiple algorithms. LIBER Quarterly: The Journal of the Association of European Research Libraries, 29(1), 1-25.

Suominen, O., & Koskenniemi, I. (2022). Annif analyzer shootout: Comparing text lemmatization methods for automated subject indexing. The Code4Lib Journal, 54. Available from:

Suominen, O., Inkinen, J., & Lehtinen, M. (2022). Annif and Finto AI: Developing and implementing automated subject indexing. JLIS.It, 13(1), 265-282.

Toepfer, M., & Seifert, C. (2020). Fusion architectures for automatic subject indexing under concept drift: Analysis and empirical results on short texts. International Journal on Digital Libraries, 21, 169-189.



How to Cite

Mukhopadhyay, P. (2023). Machine Learning and Bibliographic Data Universe: Assessing Efficacy of Backend Algorithms in Annif through Retrieval Metrics. Journal of Information and Knowledge, 60(1), 39–48.



Invited Paper

Most read articles by the same author(s)