Artificial intelligence capable of translating international guidelines for the treatment of hepatitis C into clear clinical responses consistent with the most up-to-date standards: this is the focus of an international study led by Mauro Giuffrè, PhD student at the University of Trieste (Department of Medicine, Surgery and Health Sciences) and researcher at the Yale University School of Medicine, validated by the same authors of the European guidelines for the treatment of the disease.

Hepatitis C is an infection caused by HCV, which affects the liver and can develop into chronic forms with serious complications, such as cirrhosis and hepatocellular carcinoma. According to the World Health Organisation, around 58 million people worldwide live with chronic infection and there are over 1.5 million new cases each year. WHO has set an ambitious goal of eliminating hepatitis C as a public health threat by 2030, aiming to reduce new infections by 90% and deaths by 65%.

The development of innovative tools based on artificial intelligence, such as that presented in the study by the University of Trieste, plays a key role in pursuing these objectives: Improving adherence to therapeutic guidelines and facilitating access to appropriate care even in contexts with limited resources are concrete steps that can contribute to achieving global targets.

Significant improvements in clinical accuracy

The team developed and tested two innovative approaches to specialise GPT-4 in HCV management. On the one hand, they developed a retrieval-augmented generation (RAG) system that integrates European guidelines in real time which has been tested in two variants (RAG-Top1, which retrieves the single most relevant paragraph, and RAG-Top10, which retrieves the ten most relevant paragraphs). On the other hand, they developed a supervised fine-tuning (SFT) training of the language model on the guidelines’ contents.

The results exceeded all expectations: compared to 36.6% of the GPT-4 base model, the RAG-Top10 model achieved an accuracy of 91.7% in expert evaluations, RAG-Top1 81.7% and the SFT model 71.7%, thus achieving significant improvements compared to the standard model.

A novel validation system that includes guideline extenders and clinical experts

What makes this study particularly relevant is the applied validation methodology, a new entry in the scientific literature. Two separate groups of evaluators were recruited. The first group consisted of four expert hepatologists, selected from the lead authors and chairs of the HCV guidelines of the European Association for the Study of the Liver (EASL), the leading European experts in the treatment of hepatitis C and the drafters of the international guidelines.

A second group of hepatologists was added from a tertiary reference centre (Humanitas Hospital, Rozzano), ensuring a double perspective of evaluation between guideline theorists and clinical

practitioners in the field. This approach allowed us to obtain what the researchers themselves define as ‘an evaluation that approaches the gold standard in defining the accuracy of the outputs.’

Towards responsible integration of AI in medicine

The findings open up concrete perspectives for the use of artificial intelligence in clinical decision support. Both RAG and SFT significantly improve the performance of Large Language Models (LLMs) in managing hepatitis C through guidelines, improving not only the accuracy and clarity of responses, but also the selection of therapeutic regimens in clinical scenarios. The study represents a significant step towards what the authors call ‘the safe integration of Generative Artificial Intelligence into clinical practice’, confirming the potential of specialised and expertly validated language models as concrete decision support tools in medicine, particularly valuable in highly complex contexts such as the management of chronic liver diseases. The research, presented in the article From Guidelines to Real-Time Conversation: Expert-Validated Retrieval-Augmented and Fine-Tuned GPT-4 for Hepatitis C Management, published on Liver International, was supported by Nicola Pugliese and Alessio Aghemo (Humanitas University), bioengineers from the University of Trieste Simone Kresevic and Milos Ajcevic (Department of Engineering and Architecture) and an international network of hepatologists and artificial intelligence specialists, including Dennis L. Shung (Yale), Francesco Negro (University Hospitals of Geneva), Massimo Puoti (Niguarda General Hospital; University of Milan Bicocca), Xavier Forns (Hospital Clínic Barcelona; IDIBAPS; CIBERehd) and Jean-Michel Pawlotsky (UPEC/INSERM; AP-HP Paul Brousse, Paris).