Our evidence-based R&D program has focused on systematically testing and validating AI models and reporting our findings in full.
Whilst testing AI models for literature studies we always respect the fundamental principles of an SLR – rigorous, protocol-driven, transparent evidence synthesis, free from selection bias, and with inferences weighted by study quality.
“We’ve focused our research program on using ‘machine intelligence’ to make and justify scientific decisions. Our focus has been to leverage the full capabilities of advanced LLMs, which can accurately ‘reason’ at unprecedented scale and speed.”
Dr SAIF KHARAWALA, Senior Principal Consultant & Head of Applied AI at Bridge
In Cycle 1 of our R&D program, we followed a formal testing and validation protocol of selected LLMs in the key SLR processes of title/abstract (ti/ab) screening, full-text screening, initial data extraction and table narratives. Using in-house datasets from the several hundred gold-standard SLRs we have already completed for clients, we tested both BERT and GPT models (GPT 3.5, 4, 4 turbo, 4o, o1, o1 pro, and o3) across a wide range of SLR types (across therapy areas, and with different parameters of interest, e.g., clinical-trial focused reviews, epidemiology reviews, disease burden reviews, and economic reviews).
For each SLR step, we used a unique key performance indicator (KPI) to evaluate each model – e.g., sensitivity for ti/ab and full-text screening, accuracy and sensitivity for extraction, and quality for table narratives.
Only after successful validation did we begin offering AI-enabled SLR steps to clients on live projects. As shown below, AI-enabled ti/ab screening launched in December 2023, and end-to-end, fully AI-enabled SLRs in March 2025.
Our series of white papers on our AI research can be found at the links below.
We are now applying our experience and insights in Cycle 2 of the R&D program, where we seek to further improve performance levels for literature studies, while expanding our AI capabilities into other HEOR areas.
To arrange for a deep dive into our AI research findings, contact us.
