MedLLMsPracticalGuide

<div align=center> <img src="img/Medical_LLM_logo.png" width="180px"> </div> <h2 align="center"> A Practical Guide for Medical Large Language Models </a></h2> <h5 align="center"> If you like our project, please give us a star ⭐ on GitHub for the latest update.</h5> <h5 align="center">

</h5>

This is an actively updated list of practical guide resources for Medical Large Language Models (Medical LLMs). It's based on our survey paper:

A Survey of Large Language Models in Medicine: Progress, Application, and Challenge

Hongjian Zhou^1,*, Fenglin Liu^1,*, Boyang Gu^2,*, Xinyu Zou^3,*, Jinfa Huang^4,*, Jinge Wu⁵, Yiru Li⁶, Sam S. Chen⁷, Peilin Zhou⁸, Junling Liu⁹, Yining Hua¹⁰, Chengfeng Mao¹¹, Chenyu You¹², Xian Wu¹³, Yefeng Zheng¹³, Lei Clifton¹, Zheng Li^14,†, Jiebo Luo^4,†, David A. Clifton^1,†. (*Core Contributors, †Corresponding Authors)

¹University of Oxford, ²Imperial College London, ³University of Waterloo, ⁴University of Rochester, ⁵University College London, ⁶Western University, ⁷University of Georgia, ⁸Hong Kong University of Science and Technology (Guangzhou), ⁹Alibaba, ¹⁰Harvard T.H. Chan School of Public Health, ¹¹MIT, ¹²Yale University, ¹³Tencent, ¹⁴Amazon

📣 Update News

[2024-07-10] We have updated our Version 6. Please check it out!

[2024-05-05] We have updated our Version 5. Please check it out!

[2024-03-03] We have updated our Version 4. Please check it out!

[2024-02-04] 🍻🍻🍻 Cheers, Happy Chinese New Year! We have updated our Version 3. Please check it out!

[2023-12-11] We have updated our survey Version 2. Please check it out!

[2023-11-09] We released the repository and survey Version 1.

⚡ Contributing

If you want to add your work or model to this list, please do not hesitate to email fenglin.liu@eng.ox.ac.uk and jhuang90@ur.rochester.edu or pull requests. Markdown format:

* [**Name of Conference or Journal + Year**] Paper Name. [[paper]](link) [[code]](link)

🤔 What are the Goals of the Medical LLM?

Goal 1: Surpassing Human-Level Expertise.

<div align=center> <img src="img/Medical_LLM_evolution.png" width="800px"> </div>

Goal 2: Emergent Properties of Medical LLM with the Model Size Scaling Up.

<div align=center> <img src="img/Medical_LLM_parameter_new.png" width="800px"> </div>

🤗 What is This Survey About?

This survey provides a comprehensive overview of the principles, applications, and challenges faced by LLMs in medicine. We address the following specific questions:

1. How should medical LLMs be built?
2. What are the measures for the downstream performance of medical LLMs?
3. How should medical LLMs be utilized in real-world clinical practice?
4. What challenges arise from the use of medical LLMs?
5. How should we better construct and utilize medical LLMs?

This survey aims to provide insights into the opportunities and challenges of LLMs in medicine, and serve as a practical resource for constructing effective medical LLMs.

<div align=center> <img src="img/Medical_LLM_Introduction.png" width="800px"> </div>

Table of Contents

• 📣 Update News
• ⚡ Contributing
• 🤔 What are the Goals of the Medical LLM?
• 🤗 What is This Survey About?
• Table of Contents
• 🔥 Practical Guide for Building Pipeline
  • Pre-training from Scratch
  • Fine-tuning General LLMs
  • Prompting General LLMs
• 📊 Practical Guide for Medical Data
  • Clinical Knowledge Bases
  • Pre-training Data
  • Fine-tuning Data
• 🗂️ Downstream Biomedical Tasks
  • Generative Tasks
  • Huggingface Leadboard
    • Text Summarization
    • Text Simplification
    • Question Answering
  • Discriminative Tasks
    • Entity Extraction
    • Relation Extraction
    • Text Classification
    • Natural Language Inference
    • Semantic Textual Similarity
    • Information Retrieval
• ✨ Practical Guide for Clinical Applications
  • Retrieval-augmented Generation
  • Medical Decision-Making
  • Clinical Coding
  • Clinical Report Generation
  • Medical Education
  • Medical Robotics
  • Medical Language Translation
  • Mental Health Support
• ⚔️ Practical Guide for Challenges
  • Hallucination
  • Lack of Evaluation Benchmarks and Metrics
  • Domain Data Limitations
  • New Knowledge Adaptation
  • Behavior Alignment
  • Ethical, Legal, and Safety Concerns
• 🚀 Practical Guide for Future Directions
  • Introduction of New Benchmarks
  • Interdisciplinary Collaborations
  • Multi-modal LLM
  • Medical Agents
• 👍 Acknowledgement
• 📑 Citation
• ♥️ Contributors

🔥 Practical Guide for Building Pipeline

<div align=center> <img src="img/Medical_LLMs_tree.png" width="1000px"> </div>

Pre-training from Scratch

• [Nature Medicine, 2024] BiomedGPT A generalist vision–language foundation model for diverse biomedical tasks paper
• [Nature, 2023] NYUTron Health system-scale language models are all-purpose prediction engines paper
• [Arxiv, 2023] OphGLM: Training an Ophthalmology Large Language-and-Vision Assistant based on Instructions and Dialogue. paper
• [npj Digital Medicine, 2023] GatorTronGPT: A Study of Generative Large Language Model for Medical Research and Healthcare. paper
• [Bioinformatics, 2023] MedCPT: Contrastive Pre-trained Transformers with Large-scale Pubmed Search Logs for Zero-shot Biomedical Information Retrieval. paper
• [Bioinformatics, 2022] BioGPT: Generative Pre-trained Transformer for Biomedical Text Generation and Mining. paper
• [NeurIPS, 2022] DRAGON: Deep Bidirectional Language-Knowledge Graph Pretraining. paper code
• [ACL, 2022] BioLinkBERT/LinkBERT: Pretraining Language Models with Document Links. paper code
• [npj Digital Medicine, 2022] GatorTron: A Large Language Model for Electronic Health Records. paper
• [HEALTH, 2021] PubMedBERT: Domain-specific Language Model Pretraining for Biomedical Natural Language Processing. paper
• [Bioinformatics, 2020] BioBERT: A Pre-trained Biomedical Language Representation Model for Biomedical Text Mining. paper
• [ENNLP, 2019] SciBERT: A Pretrained Language Model for Scientific Text. paper
• [NAACL Workshop, 2019] ClinicalBERT: Publicly Available Clinical BERT Embeddings. paper
• [BioNLP Workshop, 2019] BlueBERT: Transfer Learning in Biomedical Natural Language Processing: An Evaluation of BERT and ELMo on Ten Benchmarking Datasets. paper

Fine-tuning General LLMs

• [Arxiv, 2024.8] Med42-v2: A Suite of Clinical LLMs. paper Model
• [Huggingface, 2024.5] OpenBioLLM-70b: Advancing Open-source Large Language Models in Medical Domain model
• [Huggingface, 2024.5] MedLllama3 model
• [Arxiv, 2024.4] Med-Gemini Capabilities of Gemini Models in Medicine. paper
• [Arxiv, 2024.2] BioMistral A Collection of Open-Source Pretrained Large Language Models for Medical Domains. paper
• [Arxiv, 2023.12] From Beginner to Expert: Modeling Medical Knowledge into General LLMs. paper
• [Arxiv, 2023.11] Taiyi: A Bilingual Fine-Tuned Large Language Model for Diverse Biomedical Tasks. paper code
• [Arxiv, 2023.10] AlpaCare: Instruction-tuned Large Language Models for Medical Application. paper code
• [Arxiv, 2023.10] BianQue: Balancing the Questioning and Suggestion Ability of Health LLMs with Multi-turn Health Conversations Polished by ChatGPT. paper
• [Arxiv, 2023.10] Qilin-Med: Multi-stage Knowledge Injection Advanced Medical Large Language Model. paper
• [Arxiv, 2023.10] Qilin-Med-VL: Towards Chinese Large Vision-Language Model for General Healthcare. paper
• [Arxiv, 2023.10] MEDITRON-70B: Scaling Medical Pretraining for Large Language Models. paper
• [AAAI, 2024/2023.10] Med42: Evaluating Fine-Tuning Strategies for Medical LLMs: Full-Parameter vs. Parameter-Efficient Approaches. paper Model
• [Arxiv, 2023.9] CPLLM: Clinical Prediction with Large Language Models. paper
• [Arxiv, 2023.8] BioMedGPT/OpenBioMed Open Multimodal Generative Pre-trained Transformer for BioMedicine.