Protocol paper: From Chaos to Order. Augmenting Manual Article Screening with Sentence Transformers in Management Systematic Reviews

Juan A. Marin-Garcia

https://orcid.org/0000-0001-5416-3938

Spain

ROGLE - Departamento de Organización de Empresas - Universitat Politècnica de València image/svg+xml

Juan Martinez-Tomas

Spain

Universitat Politècnica de València image/svg+xml

Departamento de Organización de Empresas

Amable Juarez-Tarraga

Spain

Universitat Politècnica de València image/svg+xml

Departamento de Organización de Empresas

Cristina Santandreu-Mascarell

Spain

Universitat Politècnica de València image/svg+xml

Departamento de Organización de Empresas

|

Accepted: 2024-12-02

|

Published: 2024-12-11

DOI: https://doi.org/10.4995/wpom.22282
Funding Data

Downloads

Keywords:

protocol paper, Systematic Literature Review, Management Research, Sentence Transformers, Natural Language Processing, Article Screening, Open Science, Research Accessibility, Machine Learning, Text Embeddings, Research Methodology

Supporting agencies:

This research was not funded

Abstract:

A spanish version of the article is provided (see section before Acknowledgements)

As scientific output grows, systematic reviews have become essential yet increasingly challenging. Our approach to this protocol aims to make this process more effective, efficient and accessible to researchers worldwide, including those in developing countries.

We developed a tool to complement human judgment in the screening phase using pre-trained language models and natural language processing techniques. This tool generates text embeddings and calculates semantic similarities, prioritizing potentially relevant articles. The goal is to utilise the similarity ranking instead of reviewing articles randomly or following the relevance sort option of search engines like WOS or Scopus. Coders can start with those closest to the category/categories of interest and progressively move towards the more distant ones. This approach would save time and effort while reducing the fatigue and biases of the coders.

The models we have tested in this research are all-MiniLM-L6-v2, all-distilroberta-v1, all-mpnet-base-v2, paraphrase-multilingual-mpnet-base-v2, distiluse-base-multilingual-cased-v1, all-MiniLM-L12-v2, allenai-specter, allenai/scibert_scivocab_uncased, distilbert-base-nli-mean-tokens, roberta-base-nli-stsb-mean-tokens, distiluse-base-multilingual-cased-v2, paraphrase-multilingual-MiniLM-L12-v2, stsb-roberta-large, bert-base-nli-mean-tokens.

The method was implemented using limited computational resources and open-source software, ensuring accessibility for research teams with restricted economic resources.

Results indicate a possible reduction in screening time and improved consistency in article selection. The tool demonstrated utility in classifying relevant studies and would facilitate more comprehensive reviews.

By providing a low-cost solution, we aim to level the playing field in global research, enabling researchers from diverse economic backgrounds to participate more fully in producing scientific knowledge.

Show more Show less

References:

Aguilar-Escobar, V. G., Garrido-Vega, P., Vázquez-Rivas, P. d. V., Monzón-Moreno, A. (2024). Factors influencing nurse satisfaction with Automated Medication Dispensing Cabinets. WPOM-Working Papers on Operations Management, 15, 57-74. https://doi.org/10.4995/wpom.18935

Aguinis, H., Ramani, R. S., Alabduljader, N. (2020). Best-Practice Recommendations for Producers, Evaluators, and Users of Methodological Literature Reviews. Organizational Research Methods. https://doi.org/10.1177/1094428120943281

Ahmad, I., Ullah, K., Khan, A. (2022). The impact of green HRM on green creativity: mediating role of pro-environmental behaviors and moderating role of ethical leadership style [Article]. International Journal of Human Resource Management, 33(19), 3789-3821. https://doi.org/10.1080/09585192.2021.1931938

Alfalla-Luque, R., Luján García, D. E., Marin-Garcia, J. A. (2023). Supply chain agility and performance: evidence from a meta-analysis. International Journal of Operations & Production Management, 43(10), 1587-1633. https://doi.org/10.1108/IJOPM-05-2022-0316

Alfalla-Luque, R., Medina-Lopez, C., Dey, P.K. (2013). "Supply chain integration framework using literature review". Production Planning and Control, 24 (8-9): 800-817. https://doi.org/10.1080/09537287.2012.666870

Asmussen, C. B., Moller, C. (2019). Smart literature review: a practical topic modelling approach to exploratory literature review [Review]. Journal of Big Data, 6(1). https://doi.org/10.1186/s40537-019-0255-7

Aznar-Mas, L. E., Atarés Huerta, L., Marin-Garcia, J. A. (2023). Effectiveness of the use of open-ended questions in student evaluation of teaching in an engineering degree [Teaching evaluation, higher education, student satisfaction, teaching improvement, open-ended questions]. Journal of Industrial Engineering and Management, 16(3), 14. https://doi.org/10.3926/jiem.5620

Bayonne, E., Marin-Garcia, J.A., Alfalla-Luque, R. (2020). "Partial least squares (PLS) in Operations Management research: Insights from a systematic literature review", Journal of Industrial Engineering and Management, 13(3), pp. 565-597. https://doi.org/10.3926/jiem.3416

Becker, W. J., Belkin, L. Y., Tuskey, S. E., Conroy, S. A. (2022). Surviving remotely: How job control and loneliness during a forced shift to remote work impacted employee work behaviors and well-being [Article]. Human resource management, 61(4), 449-464. https://doi.org/10.1002/hrm.22102

Binh, D., Tung, L., Le-Minh, N. (2023). SubTST: a consolidation of sub-word latent topics and sentence transformer in semantic representation [Article]. Applied Intelligence, 53(11), 13470-13487. https://doi.org/10.1007/s10489-022-04184-x

Blei, D., Ng, A., Jordan, M. (2001). Latent Dirichlet Allocation (Vol. 3). https://doi.org/10.7551/mitpress/1120.003.0082

Booth, A., Noyes, J., Flemming, K., Moore, G., Tunçalp, Ö., Shakibazadeh, E. (2019). Formulating questions to explore complex interventions within qualitative evidence synthesis. BMJ Global Health, 4(Suppl 1), e001107. https://doi.org/10.1136/bmjgh-2018-001107

Borah, R., Brown, A. W., Capers, P. L., Kaiser, K. A. (2017). Analysis of the time and workers needed to conduct systematic reviews of medical interventions using data from the PROSPERO registry. Bmj Open, 7(2), e012545. https://doi.org/10.1136/bmjopen-2016-012545

Borenstein, M., Hedges, L. V., Higgins, J. P. T., Rothstein, H. R. (2009). Introduction to Meta-Analysis. John Wiley & Sons. https://doi.org/10.1002/9780470743386

Bruno, M., Catanese, E., De Cubellis, M., Fausti, F., Pugliese, F., Scannapieco, M., Valentino, L. (2022). Analyzing textual data through Word Embedding: experiences in Istat.

Budhwar, P., Chowdhury, S., Wood, G., Aguinis, H., Bamber, G. J., Beltran, J. R., Boselie, P., Lee Cooke, F., Decker, S., DeNisi, A., Dey, P. K., Guest, D., Knoblich, A. J., Malik, A., Paauwe, J., Papagiannidis, S., Patel, C., Pereira, V., Ren, S., Rogelberg, S., Saunders, M. N. K., Tung, R. L., Varma, A. (2023). Human resource management in the age of generative artificial intelligence: Perspectives and research directions on ChatGPT. Human Resource Management Journal, 33(3), 606-659. https://doi.org/10.1111/1748-8583.12524

Bugorski, A. T. (2014). Cosine Similarity for Article Section Classification: Using Structured Abstracts as a Proxy for an Annotated Corpus. Electronic Thesis and Dissertation Repository. 2154. https://ir.lib.uwo.ca/etd/2154

Burbano, V. C., Chiles, B. (2022). Mitigating Gig and Remote Worker Misconduct: Evidence from a Real Effort Experiment [Article]. Organization Science, 33(4), 1273-1299. https://doi.org/10.1287/orsc.2021.1488

Cha, S.-H. (2007). Comprehensive Survey on Distance/Similarity Measures Between Probability Density Functions. Int. J. Math. Model. Meth. Appl. Sci., 1.

Chandrasekaran, D., Mago, V. (2021). Evolution of Semantic Similarity-A Survey. ACM Comput. Surv., 54(2), Article 41. https://doi.org/10.1145/3440755

Cooper, H. M. (1991). Integrating Research. A guide for Literature Reviews (Vol. 2). Sage Publications, Inc.

Devika, R., Vairavasundaram, S., Mahenthar, C. S. J., Varadarajan, V., Kotecha, K. (2021). A Deep Learning Model Based on BERT and Sentence Transformer for Semantic Keyphrase Extraction on Big Social Data [Article]. IEEE Access, 9, 165252-165261. https://doi.org/10.1109/ACCESS.2021.3133651

Dhini, B. F., Girsang, A. S., Sufandi, U. U., Kurniawati, H. (2023). Automatic essay scoring for discussion forum in online learning based on semantic and keyword similarities. Asian Association of Open Universities Journal, 18(3), 262-278. https://doi.org/10.1108/AAOUJ-02-2023-0027

Do, S., Ollion, É., Shen, R. (2024). The Augmented Social Scientist: Using Sequential Transfer Learning to Annotate Millions of Texts with Human-Level Accuracy. Sociological Methods & Research, 53(3), 1167-1200. https://doi.org/10.1177/00491241221134526

Elekes, A., Schaeler, M., Boehm, K. (2017, 19-23 June 2017). On the Various Semantics of Similarity in Word Embedding Models. 2017 ACM/IEEE Joint Conference on Digital Libraries (JCDL), https://doi.org/10.1109/JCDL.2017.7991568

Friese, S., Soratto, J., Pires, D. (2018). Carrying out a computer-aided thematic content analysis with ATLAS.ti. MMG Working Papers Print(WP 18-02), 1-28. http://www.mmg.mpg.de/publications/working-papers/2018/wp-18-02/?utm_source=CleverReach&utm_medium=email&utm_campaign=16-05-2018+Newsletter+2018-03++-++May&utm_content=Mailing_12459922

Galli, C., Donos, N., Calciolari, E. (2024). Performance of 4 Pre-Trained Sentence Transformer Models in the Semantic Query of a Systematic Review Dataset on Peri-Implantitis [Review]. Information, 15(2). https://doi.org/10.3390/info15020068

Gioia, D. (2021). A Systematic Methodology for Doing Qualitative Research. The Journal of Applied Behavioral Science, 57(1), 20-29. https://doi.org/10.1177/0021886320982715

Goldberg, Y. (2022). Neural Network Methods for Natural Language Processing. Springer Cham.

Guesmi, M., Chatti, M. A., Kadhim, L., Shoeb, J., Qurat Ul, A. (2023). Semantic Interest Modeling and Content-Based Scientific Publication Recommendation Using Word Embeddings and Sentence Encoders. Multimodal Technologies and Interaction, 7(9), 91. https://doi.org/10.3390/mti7090091

Hanifi, M., Chibane, H., Houssin, R., Cavallucci, D. (2022). Problem formulation in inventive design using Doc2vec and Cosine Similarity as Artificial Intelligence methods and Scientific Papers [Article]. Engineering Applications of Artificial Intelligence, 109. https://doi.org/10.1016/j.engappai.2022.104661

Hauff, S., Felfe, J., Klug, K. (2022). High-performance work practices, employee well-being, and supportive leadership: spillover mechanisms and boundary conditions between HRM and leadership behavior [Article]. International Journal of Human Resource Management, 33(10), 2109-2137. https://doi.org/10.1080/09585192.2020.1841819

Hickman, L., Liff, J., Rottman, C., Calderwood, C. (2024). The Effects of the Training Sample Size, Ground Truth Reliability, and NLP Method on Language-Based Automatic Interview Scores' Psychometric Properties. Organizational Research Methods, 0(0), 10944281241264027. https://doi.org/10.1177/10944281241264027

Hickman, L., Thapa, S., Tay, L., Cao, M., Srinivasan, P. (2020). Text Preprocessing for Text Mining in Organizational Research: Review and Recommendations. Organizational Research Methods, 25(1), 114-146. https://doi.org/10.1177/1094428120971683

Higgins, J., Thomas, J., Chandler, J., Cumpston, M., Li, T., Page, M., Welch, V. (2021). Cochrane Handbook for Systematic Reviews of Interventions version 6.2 (updated February 2021). Cochrane. Available from https://training.cochrane.org/handbook

Houeland, C., Jordhus-Lier, D. (2022). 'Not my task': Role perceptions in a green transition among shop stewards in the Norwegian petroleum industry [Article]. Journal of Industrial Relations, 64(4), 522-543. https://doi.org/10.1177/00221856211068500

Kim, K., Kogler, D. F., Maliphol, S. (2024). Identifying interdisciplinary emergence in the science of science: combination of network analysis and BERTopic. Humanities & Social Sciences Communications, 11(1), 603. https://doi.org/10.1057/s41599-024-03044-y

Krippendorff, K. (2018). Content analysis : an introduction to its methodology. SAGE publications Inc. https://doi.org/10.4135/9781071878781

Kulkarni, A., Terpenny, J., Prabhu, V. (2023). Leveraging Active Learning for Failure Mode Acquisition. Sensors, 23(5), 2818. https://doi.org/10.3390/s23052818

Kurek, J., Latkowski, T., Bukowski, M., Świderski, B., Łępicki, M., Baranik, G., Nowak, B., Zakowicz, R., Dobrakowski, Ł. (2024). Zero-Shot Recommendation AI Models for Efficient Job-Candidate Matching in Recruitment Process. Applied Sciences, 14(6), 2601. https://doi.org/10.3390/app14062601

Levy, O., Goldberg, Y., Dagan, I. (2015). Improving distributional similarity with lessons learned from word embeddings. Transactions of the association for computational linguistics, 3, 211-225. https://doi.org/10.1162/tacl_a_00134

Liu, B. (2012). Sentiment Analysis and Opinion Mining (Vol. 5). https://doi.org/10.2200/S00416ED1V01Y201204HLT016

Losilla, J.-M., Oliveras, I., Marin-Garcia, J. A., Vives, J. (2018). Three risk of bias tools lead to opposite conclusions in observational research synthesis. Journal of Clinical Epidemiology(101), 61-72. https://doi.org/10.1016/j.jclinepi.2018.05.021

Ma, H., Su, M. (2024). Artificial stupidity and coping strategies. Organizational Dynamics, 101059. https://doi.org/10.1016/j.orgdyn.2024.101059

Manning, C. D., Raghavan, P., Schütze, H. (2008). Introduction to information retrieval. Cambridge University Press. https://doi.org/10.1017/CBO9780511809071

Marin-Garcia, J. A. (2013). What do we know about the relationship between High Involvement Work Practices and Performance? WPOM-Working Papers on Operations Management, 4(2), 01-15. https://doi.org/10.4995/wpom.v4i2.1552

Marin-Garcia, J. A. (2021). Publishing in three stages to support evidence based management practice. WPOM-Working Papers on Operations Management, 12(2), 56-95. https://doi.org/10.4995/wpom.11755

Marin-Garcia, J.A. Alfalla-Luque, R., Machuca, J.A.D. (2018). "A Triple-A supply chain measurement model: validation and analysis", International Journal of Physical Distribution & Logistics Management. Vol. 48. No. 10, pp. 976-994. https://doi.org/10.1108/IJPDLM-06-2018-0233

Marin-Garcia, J. A., Garcia-Sabater, J. J., Garcia-Sabater, J. P., Maheut, J. (2020). Protocol: Triple Diamond method for problem solving and design thinking. Rubric validation. WPOM-Working Papers on Operations Management, 11(2), 49-68. https://doi.org/10.4995/wpom.v11i2.14776

Marin-Garcia, J. A., Garcia-Sabater, J. P., Ruiz, A., Maheut, J., Garcia-Sabater, J. J. (2020). Operations Management at the service of health care management: Example of a proposal for action research to plan and schedule health resources in scenarios derived from the COVID-19 outbreak. Journal of Industrial Engineering and Management, 13(2). https://doi.org/10.3926/jiem.3190

Marin-Garcia, J. A., Martínez-Tomás, J. (2022). What does the wage structure depend on? Evidence from the national salary survey in Spain. WPOM-Working Papers on Operations Management, 13(1), 35-63. https://doi.org/10.4995/wpom.16808

Marin-Garcia, J. A., Martinez Tomas, J. (2016). Deconstructing AMO framework: a systematic review. Intangible Capital, 12(4), 1040-1087. https://doi.org/10.3926/ic.838

Marin-Garcia, J. A., Miralles Insa, C., Marin Garcia, P. (2008). Oral Presentation and Assessment Skills in Engineering Education. International Journal of Engineering Education, 24(5), 926-935. http://www.upv.es/i.grup/repositorio/own/MarinEtAl2008_IEMA_peerassessmentIJEE.pdf

Marin-Garcia, J. A., Vidal-Carreras, I., Maheut, J. (2021). A keyword taxonomy proposal for Operations Management. XII Workshop in Operations Management and Technology (ACEDEDOT OMTech 2021), Granada.

Medina-López, C., Marin-Garcia, J. A., Alfalla-Luque, R. (2010). Una propuesta metodológica para la realización de búsquedas sistemáticas de bibliografía (A methodological proposal for the systematic literature review). WPOM-Working Papers on Operations Management, 1(2), 13-30. https://doi.org/10.4995/wpom.v1i2.786

Mikolov, T., Chen, K., Corrado, G. s., Dean, J. (2013). Efficient Estimation of Word Representations in Vector Space. Proceedings of Workshop at ICLR, 2013.

Mikolov, T., Sutskever, I., Chen, K., Corrado, G. s., Dean, J. (2013). Distributed Representations of Words and Phrases and their Compositionality. Advances in Neural Information Processing Systems, 26.

Moher, D., Shamseer, L., Clarke, M., Ghersi, D., Liberati, A., Petticrew, M., Shekelle, P., Stewart, L. A., Estarli, M., Barrera, E. S. A., Martínez-Rodríguez, R., Baladia, E., Agüero, S. D., Camacho, S., Buhring, K., Herrero-López, A., Gil-González, D. M., Altman, D. G., Booth, A., Chan, A. W., Chang, S., Clifford, T., Dickersin, K., Egger, M., Gøtzsche, P. C., Grimshaw, J. M., Groves, T., Helfand, M., Higgins, J., Lasserson, T., Lau, J., Lohr, K., McGowan, J., Mulrow, C., Norton, M., Page, M., Sampson, M., Schünemann, H., Simera, I., Summerskill, W., Tetzlaff, J., Trikalinos, T. A., Tovey, D., Turner, L., Whitlock, E. (2016). Preferred reporting items for systematic review and meta-analysis protocols (PRISMA-P) 2015 statement [Article]. Revista Espanola de Nutricion Humana y Dietetica, 20(2), 148-160. https://doi.org/10.1186/2046-4053-4-1

Mora-Valentin, E.-M., Huertas-Valdivia, I., Garcia-Moreno, M.-B. (2024). Integrating the SDG into university teaching: an application in human resources subjects [Article]. WPOM-Working Papers on Operations Management, 15(1), 1-15. https://doi.org/10.4995/wpom.19824

Muñoz, S., Iglesias, C. Á. (2023). Detection of the Severity Level of Depression Signs in Text Combining a Feature-Based Framework with Distributional Representations. Applied Sciences, 13(21), 11695. https://doi.org/10.3390/app132111695

Naing, I., Soe Thandar, A., Khaing Hsu, W., Funabiki, N. (2024). A Reference Paper Collection System Using Web Scraping. Electronics, 13(14), 2700. https://doi.org/10.3390/electronics13142700

Nguyen, T. N. H., Khuu, T. P. D., Nguyen, Q. H., Nguyen, M. C. (2024). Can sustainable supply chain strategies of company enhance for mitigation of risk damages and long-term resilience? An empirical analysis for the context of COVID-19 pandemic. WPOM-Working Papers on Operations Management, 15, 112-131. https://doi.org/10.4995/wpom.21495

Nursalman, M., Kusnendar, J., Fadhila, U. F., Ieee. (2018). Implementation of K-Nearest Neighbor with Cosine Similarity for Classification Abstract International Journal of Computer Science.International Conference on Information Technology Systems and Innovation (ICITSI) [2018 international conference on information technology systems and innovation (icitsi)]. 5th International Conference on Information Technology Systems and Innovation (ICITSI), Inst Teknologi Bandung, Sch Elect Engn & Informat, INDONESIA. https://doi.org/10.1109/ICITSI.2018.8696072

O'Mara-Eves, A., Thomas, J., McNaught, J., Miwa, M., Ananiadou, S. (2015). Using text mining for study identification in systematic reviews: a systematic review of current approaches. Systematic Reviews, 4(1), 5. https://doi.org/10.1186/2046-4053-4-5

Odacioglu, E. C., Zhang, L., Allmendinger, R., Shahgholian, A. (2023). Big textual data research for operations management: topic modelling with grounded theory. International Journal of Operations & Production Management, ahead-of-print(ahead-of-print). https://doi.org/10.1108/IJOPM-03-2023-0239

Ogbonnaya, C., Brown, A. D. (2023). Editorial: Crafting review and essay articles for Human Relations. Human relations, 76(3), 365-394. https://doi.org/10.1177/00187267221148440

Page, M. J., McKenzie, J. E., Bossuyt, P. M., Boutron, I., Hoffmann, T. C., Mulrow, C. D., Shamseer, L., Tetzlaff, J. M., Akl, E. A., Brennan, S. E., Chou, R., Glanville, J., Grimshaw, J. M., Hróbjartsson, A., Lalu, M. M., Li, T., Loder, E. W., Mayo-Wilson, E., McDonald, S., McGuinness, L. A., Stewart, L. A., Thomas, J., Tricco, A. C., Welch, V. A., Whiting, P., Moher, D. (2021). The PRISMA 2020 statement: an updated guideline for reporting systematic reviews. BMJ, 372, n71. https://doi.org/10.1136/bmj.n71

Patil, R., Gudivada, A. (2024). A Review of Current Trends, Techniques, and Challenges in Large Language Models (LLMs). https://doi.org/10.20944/preprints202402.0357.v1

Perello-Marin, M. R., Ribes-Giner, G. (2014). Identifying a guiding list of high involvement practices in human resource management. WPOM-Working Papers on Operations Management, 5(1), 31-47. https://doi.org/10.4995/wpom.v5i1.1495

Perone, C., Silveira, R., Paula, T. (2018). Evaluation of sentence embeddings in downstream and linguistic probing tasks. https://doi.org/https://arxiv.org/abs/1806.06259

Rayhan, A. (2024). Advancements in Natural Language Processing: A Comprehensive Review. https://doi.org/10.13140/RG.2.2.13644.22400

Rincon, V., Zorrilla, P. (2017). Management Protocol: Entrepreneurship in the area of Marketing. Comparing PBL vs active lectures [Article]. WPOM-Working Papers on Operations Management, 8(1), 1-8. https://doi.org/10.4995/wpom.v8i1.6470

Rincón, V., Zorrilla, P., Marin-Garcia, J. A. (2023). The impact of active learning on entrepreneurial capacity [Entrepreneurship, creativity, innovation, competences, marketing]. Intangible Capital, 19(4), 16. https://doi.org/10.3926/ic.2297

Sanchez-Cazorla, A., Alfalla-Luque, R., Irimia-Diéguez, A. (2016). "Risk identification in Megaprojects as a crucial phase of Risk Management: a literature review", Project Management Journal. Vol. 47, No. 6, 75-93. https://doi.org/10.1177/875697281604700606

Santandreu Mascarell, C., Juarez-Tarraga, A., Marin-Garcia Juan, A. (2024). What do we need to research in the future on high involvement work practices? XIII International Workshop on HRM, Sevilla on September 19-20, 2024.

Saunders, M., Lewis, P., Thornhill, A. (2016). Research methods for business students, 7/e. Pearson Education.

Scarpino, I., Zucco, C., Vallelunga, R., Luzza, F., Cannataro, M. (2022). Investigating Topic Modeling Techniques to Extract Meaningful Insights in Italian Long COVID Narration. BioTech, 11(3), 41. https://doi.org/10.3390/biotech11030041

Sebastiani, F. (2002). Machine learning in automated text categorization. ACM Comput. Surv., 34(1), 1-47. https://doi.org/10.1145/505282.505283

Sidorov, G., Gelbukh, A., Gómez-Adorno, H., Pinto, D. (2014). Soft Similarity and Soft Cosine Measure: Similarity of Features in Vector Space Model. Computación y Sistemas, 18(3), 491-504. https://doi.org/10.13053/cys-18-3-2043

Singh, I., Scarton, C., Bontcheva, K. (2021). Multistage BiCross encoder for multilingual access to COVID-19 health information. PLoS ONE, 16(9). https://doi.org/10.1371/journal.pone.0256874

Song, W., Yu, H., Qu, Q. (2021). High involvement work systems and organizational performance: the role of knowledge combination capability and interaction orientation [Article]. International Journal of Human Resource Management, 32(7), 1566-1590. https://doi.org/10.1080/09585192.2018.1539863

Speer, A. B., Perrotta, J., Kordsmeyer, T. L. (2024). Taking It Easy: Off-the-Shelf Versus Fine-Tuned Supervised Modeling of Performance Appraisal Text. Organizational Research Methods, 0(0), 10944281241271249. https://doi.org/10.1177/10944281241271249

Su, Z., Wang, D., Miao, C., Cui, L. (2023). Multi-Aspect Explainable Inductive Relation Prediction by Sentence Transformer [preprint]. ArXiv. https://doi.org/10.1609/aaai.v37i5.25803

Tranfield, D., Denyer, D., Smart, P. (2003). Towards a methodology for developing evidence-informed management knowledge by means of systematic review. British Journal of Management, 14, 207--222. https://doi.org/10.1111/1467-8551.00375

Troxler, A., Schelldorfer, J. (2024). Actuarial applications of natural language processing using transformers: Case studies for using text features in an actuarial context. British Actuarial Journal, 29. https://doi.org/10.1017/S1357321724000023

Van Rhee, H. J., Suurmond, R., Hak, T. (2018). User manual for Meta-Essentials: Workbooks for meta-analysis (Version 1.4). Erasmus Research Institute of Management. https://www.erim.eur.nl/research-support/meta-essentials

Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A., Kaiser, L., Polosukhin, I. (2017). Attention Is All You Need. https://doi.org/10.48550/arXiv.1706.03762

Wang, Y., Liu, S., Afzal, N., Rastegar-Mojarad, M., Wang, L., Shen, F., Kingsbury, P., Liu, H. (2018). A comparison of word embeddings for the biomedical natural language processing [Article]. Journal of Biomedical Informatics, 87, 12-20. https://doi.org/10.1016/j.jbi.2018.09.008

Wu, L., Ali, S., Ali, H., Brock, T., Xu, J., Weida, T. (2022). NeuroCORD: A Language Model to Facilitate COVID-19-Associated Neurological Disorder Studies. International Journal of Environmental Research and Public Health, 19(16), 9974. https://doi.org/10.3390/ijerph19169974

Xia, P., Zhang, L., Li, F. (2015). Learning similarity with cosine similarity ensemble [Article]. Information Sciences, 307, 39-52. https://doi.org/10.1016/j.ins.2015.02.024

Young, T., Hazarika, D., Poria, S., Cambria, E. (2018). Recent Trends in Deep Learning Based Natural Language Processing [Review Article]. IEEE Computational Intelligence Magazine, 13(3), 55-75. https://doi.org/10.1109/MCI.2018.2840738

Show more Show less