A Comparison of Six Global Ranking Systems Considering Weight Distribution and Indicator Types by Correlation Analysis

Sumita Ghose; Md. Yeosuf Akhtar

doi:10.17821/srels/2023/v60i5/171125

Authors

Sumita Ghose
Department of Library and Information Science, RKDF University, Ranchi – 834004, Jharkhand
Md. Yeosuf Akhtar
Department of Library and Information Science, RKDF University, Ranchi – 834004, Jharkhand

DOI:

https://doi.org/10.17821/srels/2023/v60i5/171125

Keywords:

Confidence Level, Higher Education Institutions, Karl Pearson’s Statistical Measure, Mapping of the Indicators, Ranking Systems, Spearman’s Rank Correlation Method

Abstract

The study examines and compares the indicators used by the six global ranking systems, as well as the weights assigned to the indicators used by each ranking system. In the comparison process, the inter-ranking correlation between the ranking systems is determined by Spearman’s rank correlation method to understand primarily the similarities between different ranking systems based on rank or positions held by different institutions. The study frames all the indicators, considering various ranking systems, in a common ranking framework for establishing the co-relation matrix. The framing of seven distinct ranking characteristics or indications aids in understanding the weighting scheme used by the six ranking methods. The work then further investigates the reasons for differing Spearman’s ranking correlation values by exploring different ways of calculating the ranking indices adopted by different ranking agencies. The weight distribution across the indicators is another factor which is required to be taken into consideration while observing the correlation between the two ranking systems. Karl Pearson’s statistical measure is applied to find the similarities of the weights associated with each indicator in different raking systems. The outcome of this research provides a comprehensive view of countries like India which are participating in different international ranking systems to improve their ranking while taking part in various ranking systems. Different test cases and in-depth analyses show that it is not an easy task for the institutes to perform better in all the ranking systems.

Downloads

Download data is not yet available.

Metrics

Metrics Loading ...

References

Aguillo, I. F., Bar-Ilan, J., Levene, M., and Ortega, J. L. (2010). Comparing university rankings. Scientometrics, 85(1), 243-256.

Allam, M. (2019). International rankings and Indian institutions of higher education: An analysis. Universal Review, 10(7), 45-60.

Atanbiyi, G., Aribisala, B., Nyaaku, O., and Olabanjo, O. (2020). Development of objective ranking system for tertiary institutions in developing nations. Journal of Research and Review in Science, 7, 39-53. https://doi.org/10.36108/ jrrslasu/0202.70.0170

Bar-Ilan, J., Levene, M., and Lin, A. (2007). Some measures for comparing citation databases. Journal of Informetrics, 1(1), 26-34.

Bandyopadhyay, S., and Sharma, A. (2020). Ranking frameworks and quality of higher educational institutions: Some missing links. https://doi.org/10.2139/ssrn.3835899

Chen, K., and Liao, P. (2012). A comparative study on world university rankings: A bibliometric survey. Scientometrics, 92(1), 89-103. https://doi.org/10.1007/s11192-012-0724-7

Cakir, M. P., Acarturk, C., Alasehir, O., and Cilingir, C. (2015). A comparative analysis of global and national university ranking systems. Scientometrics, 103, 813-848. https://doi. org/10.1007/s11192-015-1586-6

Hou,Y., and Jacob, W. J. (2017). What contributes more to the ranking of higher education institutions? A comparison of three world university rankings. The International Education Journal: Comparative Perspectives, 16(4), 29-46. https://openjournals.library.sydney.edu.au/index.php/IEJ

Joorel, J. P. S., Kumar, A., Solanki, H., Raja, V., Shah, D., Pradhan, P., Trivedi, K., and Singh. (2021). Five years of India rankings (2016–2020): An evolutionary study. DESIDOC Journal of Library & Information Technology, 41(1), 42-48. https://doi.org/10.14429/djlit.41.1.16671

Kumar, A., Singh, K., and Siwach, A. K. (2021), NIRF India rankings 2020: Analyzing the ranking parameters and score of top 100 universities. DESIDOC Journal of Library and Information Technology, 41(5), 385-390. https://doi. org/10.14429/djlit.41.5.16452

Lukman, R., Krajnc, D., and Glavic, P. (2010). University ranking using research, educational and environmental indicators. Journal of Cleaner Production, 18(7), 619-628. https://doi.org/10.1016/j.jclepro.2009.09.015

McCormick, A. C. (2013). Classifying higher education institutions: Lessons from the Carnegie Classification. Center for Postsecondary Research, Indiana University Bloomington, USA, 50(1), 65-75. https://doi.org/10.7764/PEL.50.1.2013.6

Nassa, A. K. and Arora, J. (2021). Ranking of Institutions of higher education. DESIDOC Journal of Library and Information Technology, 1(1), 3-4. https://doi. org/10.14429/djlit.41.1.16757

Obasi, I. N. (2016). World university rankings in a marketdriven knowledge society. Implications for African Universities. University of Botswana, Botswana.

Salmi, J., and Saroyan, A. (2007). League tables as policy instruments: Uses and misuses. Journal of Higher Education Policy and Management, 19(2), 1-38. ttps://doi. org/10.1057/palgrave.hep.8300154

Sayed, O. H. (2019). Critical treatise on university ranking systems. Open Journal of Social Sciences, 7, 39-51. https:// doi.org/10.4236/jss.2019.712004

Shehatt, I. and Mahmood, K.(2016). Correlation among top 100 universities in the major six global rankings: Policy implications. Scientometrics, 109, 1231-1254. https://doi. org/10.1007/s11192-016-2065-4

Singh, P., Joorel, J. P. S., Solanki, H., Kumar, A., and Trivedi, K. (2021), Ranking of Indian research-intensive higher education institutions using Multiple ranking methodologies: A correlation analysis. DESIDOC Journal of Library and Information Technology, 41(1), 49-53. https://doi.org/10.14429/djlit.41.1.16683

Pitman, T., Edwards, D., Liang-Cheng, Z., Paul, K., and Julie, M. (2020). Constructing a ranking of higher education institutions based on equity: Is it possible or desirable? Higher Education. https://doi.org/10.1007/s10734-019-00487-0

Webster, D. S. (1992). Reputational rankings of colleges, universities, and individual disciplines and fields of study, from their beginnings to the present. (J. C. Smart (Ed.)). Higher Education: Handbook of Theory and Research, New York, Agathon Press, pp. 234-304.

Official Websites of Ranking Systems

Shanghai Ranking’s Academic Ranking of World Universities Methodology 2022 https://www.shanghairanking.com/ methodology/arwu/2022. (Accessed on 06-09-2022)

THE WUR 2022 Methodology: https://www. timeshighereducation.com/world-university-rankings/ world-universityrankings-2022-methodology. (Accessed on 06-09-2022)

US News Best Global University Rankings. (2022). US New https://www.usnews.com/education/bestglobaluniversities/ articles/methodology. (Accessed on 11.08.2022).

NTU Ranking. (2022). National Taiwan University. http:// nturanking.csti.tw/. (Access on 11.08.2022). Round University Ranking. (2022). https://roundranking.com/. (Accessed on 11.08.2022).

QS World University Rankings: https://support.qs.com/hc/ en-gb/articles/4405955370898-QS-World-University- Rankings. (Accessed on 06-09-2022).