Use of AI in Systematic Literature Reviews

by
Articles

Use of AI in Systematic Literature Reviews

Systematic literature reviews (SLRs) generate comprehensive, evidence-based outputs in response to specific research questions using a structured, yet transparent approach. In clinical research, SLRs can support the holistic evaluation of the efficacy, safety, and cost-effectiveness of a medicinal product by combining findings from multiple clinical studies. These reviews provide clinicians, regulators, policy makers, economists,  epidemiologists, and health technology assessment (HTA) bodies with more meaningful information than typical review articles.

An SLR starts with creating a search strategy which is free of selection bias. This strategy is used to find relevant research articles for review and analysis. However, in this digital era, where enormous amounts of data are just a click away, conducting an SLR can be a daunting and laborious task. Literature searches can retrieve thousands of publications and therefore manually reviewing and analysing these can be time-consuming. By the time an SLR is published, the results may be outdated and out of context.

Artificial intelligence (AI) can significantly improve the efficiency and accuracy of the SLRs. Typically, an SLR is conducted using the following steps:

AI can support some of these steps very efficiently, as detailed below:

Automated Search and Screening

The predominant use of AI in SLRs is to help streamline the initial steps of the process by automating the search for relevant literature and screening articles based on predefined eligibility criteria. Search engines now utilise AI, especially by strengthening Retrieval-Augmented Generation (RAG) frameworks through large language models. These can support the preparation of a search with complex queries that surpasses the capabilities of a conventional keywords-based literature search.

Typically, machine learning classifiers are used to find more pertinent articles. These classifiers are trained on a first set of user-selected papers, and then reiteratively use the automatic classifications and retrain the classifier to better identify further relevant literature.

AI techniques employed by automated tools use an article’s title, abstract, or the full text of papers as inputs, and then Natural Language Processing (NLP) algorithms analyse abstracts, titles, and keywords to determine their relevance to the research topic. The AI techniques can also incorporate a statistical selection process to detect the key terms characterising each cluster by scoring each citation based on the number of keywords present, enabling the screener to make more efficient decisions on their relevance. The resulting clusters emphasise the most representative terms and thereby better support the inclusion or omission of a publication from the analysis.

Data Extraction and Evidence Generation

In health research, researchers apply various protocols for literature review depending on the type of report to be generated. These include PICO (population, intervention, comparison, outcome), PCC (population, context, concept), PICODR (elements of PICO plus duration and results), PIBOSO (population, intervention, background, outcome, study design, and other). 

Using AI, information is extracted from a predefined set of fields for interventional, diagnostic, or prognostic SLRs. For instance, NLP algorithms can extract key information such as study methodologies, findings, and statistical data, which can then be synthesised and analysed to draw meaningful conclusions. Knowledge graphs are produced that improve research paper classification and characterisation. AI technologies use a domain ontology to organise the data, which offers a formal description of the variable types and their association with each other.

Quality Assessment

It is important to minimise selection bias and maximise the external and internal validity of the publications selected for an SLR. To do this, systematic errors should be reduced to the largest possible extent.

A quality assessment provides evidence of the overall strength and reliability of an SLR. AI can help assess the quality of included studies by analysing factors such as study design, sample size, and methodology.

Machine learning algorithms can be trained on existing datasets to recognise patterns indicative of high-quality research, helping researchers to evaluate the reliability of the evidence more efficiently. Most validated checklists, such as PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) recommend evaluation of the bias in literature search and selection by an independent reviewer. Therefore, combining a manual quality check with automated screening is important to understand any gaps and discrepancies, which can be resolved by evaluating the conflicts between the screener’s and the reviewer's decisions.

Analysis, Data Visualization, and Preparation of the Report

AI-powered tools can support the meta-analysis of data extracted from multiple studies, enabling researchers to quantitatively synthesize findings and assess the overall effect sizes. AI can facilitate the organisation and categorisation of large volumes of literature through semantic analysis and clustering techniques. By identifying common themes and relationships among studies, AI algorithms can help researchers gain deeper insights into the existing literature, evaluate the current status of the topic of research, and identify gaps or areas for further investigation.

Additionally, AI-driven visualization techniques can be used to present complex information in a more accessible and comprehensible manner, facilitating understanding and decision-making.

AI technology uses algorithms and models to find patterns and trends, outliers in data, and correlations amongst diverse variables. Insights and recommendations generated from SLR data may help researchers understand the implications of gaps in current knowledge, processes, research, and policies.  By incorporating feedback from researchers and updating algorithms based on new data, AI can continuously enhance the accuracy and efficiency of SLRs and improve the quality of research outputs.

Allied uses of Artificial Intelligence

An SLR report usually includes many abbreviated terms and acronyms and these must all be listed. AI can be used automate the creation of the abbreviations list. In addition, AI can also assist automated quality check of the data presented in an SLR.

Overall, the integration of AI into SLR looks promising, especially for enhancing the efficiency, accuracy, and comprehensiveness of the review process, enabling researchers to leverage the vast amount of available literature more effectively.

To know more…

Write to us, if you want to know more about our customised and cost-effective AI solutions for SLRs, targeted literature reviews, and meta-analyses at: [email protected]