Awesome-Language-Model-on-Graphs

A curated list of papers and resources about large language models (LLMs) on graphs based on our survey paper: Large Language Models on Graphs: A Comprehensive Survey.

This repo will be continuously updated. Don't forget to star <img src="./fig/star.svg" width="15" height="15" /> it and keep tuned!

Please cite the paper in Citations if you find the resource helpful for your research. Thanks!

<p align="center"> <img src="./fig/intro.svg" width="90%" style="align:center;"/> </p>

Why LLMs on graphs?

Large language models (LLMs), such as ChatGPT and LLaMA, are creating significant advancements in natural language processing, due to their strong text encoding/decoding ability and newly found emergent capability (e.g., reasoning). While LLMs are mainly designed to process pure texts, there are many real-world scenarios where text data are associated with rich structure information in the form of graphs (e.g., academic networks, and e-commerce networks) or scenarios where graph data are captioned with rich textual information (e.g., molecules with descriptions). Besides, although LLMs have shown their pure text-based reasoning ability, it is underexplored whether such ability can be generalized to graph scenarios (i.e., graph-based reasoning). In this paper, we provide a comprehensive review of scenarios and techniques related to large language models on graphs.

Contents

• Awesome-Language-Model-on-Graphs
  • Why LLMs on graphs?
  • Contents
    • Keywords Convention
  • Perspectives
  • Pure Graphs
    • Datasets
    • Direct Answering
    • Heuristic Reasoning
    • Algorithmic Reasoning
  • Text-Attributed Graphs
    • Datasets
    • LLM as Predictor (Node)
      • Graph As Sequence (Node)
      • Graph-Empowered LLM (Node)
      • Graph-Aware LLM Finetuning
    • LLM as Encoder
      • Optimization
      • Data Augmentation
      <!-- - [Knowledge Distillation](#knowledge-distillation) -->
      • Efficiency
    • LLM as Aligner (Node)
      • Prediction Alignment
      • Latent Space Alignment (Node)
  • Text-Paired Graphs (Molecules)
    • Datasets
    • LLM as Predictor (Graph)
      • Graph As Sequence
      • Graph-Empowered LLM (Graph)
    • LLM as Aligner (Graph)
      • Latent Space Alignment (Graph)
  • Contribution
  • Citations

Keywords Convention

The Transformer architecture used in the work, e.g., EncoderOnly, DecoderOnly, EncoderDecoder.

The size of the large language model, e.g., medium (i.e., less than 1B parameters), LLM (i.e., more than 1B parameters).

Perspectives

1. Unifying Large Language Models and Knowledge Graphs: A Roadmap. preprint

   Shirui Pan, Linhao Luo, Yufei Wang, Chen Chen, Jiapu Wang, Xindong Wu [PDF], 2023.6

2. Integrating Graphs with Large Language Models: Methods and Prospects preprint

   Shirui Pan, Yizhen Zheng, Yixin Liu [PDF], 2023.10

3. Towards graph foundation models: A survey and beyond. preprint

   Jiawei Liu, Cheng Yang, Zhiyuan Lu, Junze Chen, Yibo Li, Mengmei Zhang, Ting Bai, Yuan Fang, Lichao Sun, Philip S. Yu, Chuan Shi. [PDF], 2023.10

4. A Survey of Graph Meets Large Language Model: Progress and Future Directions. preprint

   Yuhan Li, Zhixun Li, Peisong Wang, Jia Li, Xiangguo Sun, Hong Cheng, Jeffrey Xu Yu. [PDF], 2023.11

Pure Graphs

<img src="./fig/star.svg" width="15" height="15" /> Datasets

Table 3 in our survey paper Large Language Models on Graphs: A Comprehensive Survey.

<p align="center"> <img src="./fig/puregraph-data.jpg" width="90%" style="align:center;"/> </p>

<img src="./fig/star.svg" width="15" height="15" /> Direct Answering

1. Can Language Models Solve Graph Problems in Natural Language? preprint

   Heng Wang, Shangbin Feng, Tianxing He, Zhaoxuan Tan, Xiaochuang Han, Yulia Tsvetkov. [PDF] [Code], 2023.5,

2. GPT4Graph: Can Large Language Models Understand Graph Structured Data ? An Empirical Evaluation and Benchmarking. preprint

   Jiayan Guo, Lun Du, Hengyu Liu, Mengyu Zhou, Xinyi He, Shi Han. [PDF], 2023.5,

3. Evaluating Large Language Models on Graphs: Performance Insights and Comparative Analysis. preprint

   Chang Liu, Bo Wu. [PDF] [Code], 2023.8, [PDF], 2023.5,

4. Talk Like A Graph: Encoding Graphs For Large Language Models. preprint

   Bahare Fatemi, Jonathan Halcrow, Bryan Perozzi. [PDF], 2023.10,

5. GraphLLM: Boosting Graph Reasoning Ability of Large Language Model. preprint

   Ziwei Chai, Tianjie Zhang, Liang Wu, Kaiqiao Han, Xiaohai Hu, Xuanwen Huang, Yang Yang. [PDF] [Code], 2023.10,

6. LLM4DyG: Can Large Language Models Solve Problems on Dynamic Graphs?. preprint

   Zeyang Zhang, Xin Wang, Ziwei Zhang, Haoyang Li, Yijian Qin, Simin Wu, Wenwu Zhu [PDF] [Code], 2023.10,

7. Which Modality should I use - Text, Motif, or Image? : Understanding Graphs with Large Language Models. preprint

   Debarati Das, Ishaan Gupta, Jaideep Srivastava, Dongyeop Kang [PDF] [Code], 2023.11,

8. GraphArena: Benchmarking Large Language Models on Graph Computational Problems. preprint

   Jianheng Tang, Qifan Zhang, Yuhan Li, Jia Li [PDF] [Code], 2024.7,

<img src="./fig/star.svg" width="15" height="15" /> Heuristic Reasoning

1. StructGPT: A General Framework for Large Language Model to Reason over Structured Data. preprint

   Jinhao Jiang, Kun Zhou, Zican Dong, Keming Ye, Wayne Xin Zhao, Ji-Rong Wen. [PDF] [Code], 2023.5,

2. Think-on-Graph: Deep and Responsible Reasoning of Large Language Model on Knowledge Graph. preprint

   Jiashuo Sun, Chengjin Xu, Lumingyuan Tang, Saizhuo Wang, Chen Lin, Yeyun Gong, Lionel M. Ni, Heung-Yeung Shum, Jian Guo. [PDF] [Code], 2023.7,

3. Exploring Large Language Model for Graph Data Understanding in Online Job Recommendations. preprint

   Likang Wu, Zhaopeng Qiu, Zhi Zheng, Hengshu Zhu, Enhong Chen. [PDF] [Code], 2023.7,

4. Knowledge Graph Prompting for Multi-Document Question Answering. AAAI2024

   Yu Wang, Nedim Lipka, Ryan Rossi, Alex Siu, Ruiyi Zhang, Tyler Derr. [PDF] [Code], 2023.8,

5. ChatRule: Mining Logical Rules with Large Language Models for Knowledge Graph Reasoning. preprint

   Linhao Luo, Jiaxin Ju, Bo Xiong, Yuan-Fang Li, Gholamreza Haffari, Shirui Pan. [PDF] [Code], 2023.9,

6. Reasoning on Graphs: Faithful and Interpretable Large Language Model Reasoning. preprint

   Linhao Luo, Yuan-Fang Li, Gholamreza Haffari, Shirui Pan. [PDF] [Code], 2023.10,

7. Thought Propagation: An Analogical Approach to Complex Reasoning with Large Language Models. preprint

   Junchi Yu, Ran He, Rex Ying. [PDF], 2023.10,

8. Large Language Models Can Learn Temporal Reasoning. preprint

   Siheng Xiong, Ali Payani, Ramana Kompella, Faramarz Fekri. [PDF], 2024.1,

9. Exploring the Limitations of Graph Reasoning in Large Language Models. preprint

   Palaash Agrawal, Shavak Vasania, Cheston Tan. [PDF], 2024.2,

10. Rendering Graphs for Graph Reasoning in Multimodal Large Language Models. preprint

    Yanbin Wei, Shuai Fu, Weisen Jiang, James T. Kwok, Yu Zhang. [PDF], 2024.2,

11. Graph-enhanced Large Language Models in Asynchronous Plan Reasoning. preprint

    Fangru Lin, Emanuele La Malfa, Valentin Hofmann, Elle Michelle Yang, Anthony Cohn, Janet B. Pierrehumbert. [PDF], 2024.2,

12. Microstructures and Accuracy of Graph Recall by Large Language Models. preprint

    Yanbang Wang, Hejie Cui, Jon Kleinberg. [PDF], 2024.2,

13. Structure Guided Prompt: Instructing Large Language Model in Multi-Step Reasoning by Exploring Graph Structure of the Text. preprint

    Kewei Cheng, Nesreen K. Ahmed, Theodore Willke, Yizhou Sun. [PDF], 2024.2,

14. GraphInstruct: Empowering Large Language Models with Graph Understanding and Reasoning Capability. preprint

    Zihan Luo, Xiran Song, Hong Huang, Jianxun Lian, Chenhao Zhang, Jinqi Jiang, Xing Xie, Hai Jin. [PDF], 2024.3,

15. Call Me When Necessary: LLMs can Efficiently and Faithfully Reason over Structured Environments. preprint

    Sitao Cheng, Ziyuan Zhuang, Yong Xu, Fangkai Yang, Chaoyun Zhang, Xiaoting Qin, Xiang Huang, Ling Chen, Qingwei Lin, Dongmei Zhang, Saravan Rajmohan, Qi Zhang. [PDF], 2024.3,

16. **Exploring the