Awesome-Hyperbolic-Representation-and-Deep-Learning

Introduction

Recently, hyperbolic spaces have emerged as a promising alternative for processing data with a tree-like structure or power-law distribution, owing to its exponential growth property and tree-likeness prior. Different from the Euclidean space, which expands polynomially, the hyperbolic space grows exponentially which makes it gain natural advantages in abstracting tree-like or scale-free data with hierarchical organizations. In this repository, we categorize papers related to hyperbolic representation learning into different types to facilitate researcher studies and to promote the development of the community. We will keep updating this repository with the latest research developments. We are aware that there will inevitably be some mistakes and oversights, so if you have any questions or suggestions, please feel free to contact us (menglin.yang[@]outlook.com).

<table> <tr><td colspan="2"><a href="#latest-update", p style="color:#B22222">1. Lastest Update</a></td></tr> <tr><td colspan="2"><a href="#surveys-books-tools-tutorials", p style="color:#B22222">2. Surveys, Books, Tools and Tutorials</a></td></tr> <tr> <td>&ensp;<a href="#surveys">2.1 Surveys</a></td> <td>&ensp;<a href="#books">2.2 Books </a></td> </tr> <tr> <td>&ensp;<a href="#tools">2.3 Tools </a></td> <td>&ensp;<a href="#tutorials">2.4 Tutorials</a></td> </tr> <tr><td colspan="2"><a href="#methods-and-models", p style="color:#B22222">3. Methods and Models</a></td></tr> <tr> <td>&ensp;<a href="#hyperbolic-shallow-model">3.1 Hyperbolic Shallow Model</a></td> <td>&ensp;<a href="#hyperbolic-neural-network">3.2 Hyperbolic Neural Network</a></td> </tr> <tr> <td>&ensp;<a href="#hyperbolic-graph-neural-network">3.3 Hyperbolic Graph Neural Network</a></td> <td>&ensp;<a href="#mixed-curvature-learning">3.4 Mixed Curvature Learning</a></td> </tr> <tr> <td>&ensp;<a href="#ultrahyperbolic-learning">3.5 Ultrahyperbolic Learning</a></td> <td>&ensp;<a href="#hyperbolic-operations">3.6 Hyperbolic Operations</a></td> </tr> <tr> <td>&ensp;<a href="#hyperbolic-generation-models">3.7 Hyperbolic Generation Models</a></td> <td>&ensp;<a href="#llm-and-hyperbolic-space">3.8 LLM && Hyperbolic Space</a></td> </tr> <tr><td colspan="2"><a href="#applications", p style="color:#B22222">4. Applications</a></td></tr> <tr> <td>&ensp;<a href="#recommender-systems">4.1 Recommender Systems</a></td> <td>&ensp;<a href="#knowledge-graphs">4.2 Knowledge Graphs</a></td> </tr> <tr> <td>&ensp;<a href="#molecular-learning">4.3 Molecular Learning </a></td> <td>&ensp;<a href="#dynamic-graphs">4.4 Dynamic Graphs</a></td> </tr> <tr> <td>&ensp;<a href="#code-representation">4.5 Code Representation</a></td> <td>&ensp;<a href="#graph-embeddings">4.6 Graph Embedding</a></td> </tr> <tr> <td>&ensp;<a href="#word-embeddings">4.7 Word Embedding</a></td> <td>&ensp;<a href="#multi-label-learning">4.8 Multi-label Learning</a></td> </tr> <tr> <td>&ensp;<a href="#computer-vision">4.9 Computer Vision</a></td> <td>&ensp;<a href="#natural-language-processing">4.10 Natural Language Processing</a></td> </tr> </table>

Hyperbolic Slack Group

• Slack: https://join.slack.com/t/hyperboliclearning/shared_invite/zt-1qcqgtwfr-HpsRSzDhvkAEal6dOnKDvA

✨New❗️(July 4, 2024)

1. Hypformer: Exploring Efficient Hyperbolic Transformer Fully in Hyperbolic Space, KDD 2024

2. Hyperbolicity Measures “Democracy” in Real-World Networks, Phys. Rev. E 2015

3. The Numerical Stability of Hyperbolic Representation Learning, ICML 2023

4. Fully Hyperbolic Convolutional Neural Networks for Computer Vision, ICLR 2024

5. The Dark Side of the Hyperbolic Moon, ICLR 2024

6. Leveraging Hyperbolic Embeddings for Coarse-to-Fine Robot Design, ICLR 2024

7. Fast Hyperboloid Decision Tree Algorithms, ICLR 2024

8. Ultra-sparse network advantage in deep learning via Cannistraci-Hebb brain-inspired training with hyperbolic meta-deep community-layered epitopology, ICLR 2024

9. Matrix Manifold Neural Networks++, ICLR 2024

10. Hyperbolic VAE via Latent Gaussian Distributions, NeurIPS 2023
    Seunghyuk Cho, Juyong Lee, Dongwoo Kim

11. Hyperbolic Space with Hierarchical Margin Boosts Fine-Grained Learning from Coarse Labels, NeurIPS, 2023
    Shu-Lin Xu, Yifan Sun, Faen Zhang, Anqi Xu, Xiu-Shen Wei, Yi Yang

12. Hyperbolic Graph Neural Networks at Scale: A Meta Learning Approach, NeurIPS 2023
    Nurendra Choudhary, Nikhil Rao, Chandan K. Reddy

13. Fitting trees to $\ell_1$-hyperbolic distances, NeurIPS 2023
    Joon-Hyeok Yim, Anna Gilbert

14. Leveraging Hyperbolic Embeddings for Coarse-to-Fine Robot Design, arxiv 2023
    Heng Dong, Junyu Zhang, Chongjie Zhang

15. Alignment and Outer Shell Isotropy for Hyperbolic Graph Contrastive Learning, arxiv 2023
    Yifei Zhang, Hao Zhu, Jiahong Liu, Piotr Koniusz, Irwin King

16. Riemannian Residual Neural Networks, arxiv 2023
    Isay Katsman, Eric Ming Chen, Sidhanth Holalkere, Anna Asch, Aaron Lou, Ser-Nam Lim, Christopher De Sa

17. Tempered Calculus for ML: Application to Hyperbolic Model Embedding, arxiv 2024
    Richard Nock, Ehsan Amid, Frank Nielsen, Alexander Soen, Manfred K. Warmuth

Surveys, Books, Tools, Tutorials

Surveys

1. Hyperbolic Deep Learning in Computer Vision: A Survey, arxiv 2023
   Pascal Mettes, Mina Ghadimi Atigh, Martin Keller-Ressel, Jeffrey Gu, Serena Yeung

2. Hyperbolic Graph Neural Networks: A Review of Methods and Application, arxiv 2022. GitHub
   Menglin Yang, Min Zhou, Zhihao Li, Jiahong Liu, Lujia Pan, Hui Xiong, Irwin King

3. Hyperbolic Deep Neural Networks: A Survey, TPAMI 2022. GitHub
   Wei Peng, Tuomas Varanka, Abdelrahman Mostafa, Henglin Shi, Guoying Zhao

4. Hyperbolic Geometry in Computer Vision: A Survey, arxiv 2023.
   Pengfei Fang, Mehrtash Harandi, Trung Le, Dinh Phung

Books

1. An Introduction to Geometric Topology, 2022
   Bruno Martelli

2. Hyperbolic Geometry, 2020.
   Brice Loustau

3. Manifolds and Differential Geometry, 2009.
   Jeffrey M. Lee

4. Introduction to Hyperbolic Geometry, 1995.
   A Ramsay, RD Richtmyer

Tools

1. Geoopt: Riemannian Adaptive Optimization Methods ICLR 2019
   Max Kochurov and Rasul Karimov and Serge Kozlukov

2. Curvature Learning Framework
   Alibaba

3. GraphZoo: A Development Toolkit for Graph Neural Networks with Hyperbolic Geometries WWW 2022
   Anoushka Vyas, Nurendra Choudhary, Mehrdad Khatir, Chandan K. Reddy

4. HypLL: The Hyperbolic Learning Library, GitHub
   Max van Spengler, Philipp Wirth, Pascal Mettes

Tutorials

1. Hyperbolic Deep Learning for Computer Vision
   Pascal Mettes, Max van Spengler, Yunhui Guo, Stella Yu

2. Hyperbolic networks: Theory, Architecture and Applications
   Nurendra Choudhary, Nikhil Rao, Karthik Subbian, Srinivasan H. Sengamedu, Chandan Reddy

3. Hyperbolic Graph Neural Networks: A Tutorial on Methods and Applications, KDD 2023
   Min Zhou, Menglin Yang, Bo Xiong, Hui Xiong, Irwin King

4. Hyperbolic Representation Learning for Computer Vision. Tutorial 2022
   Pascal Mettes, Mina Ghadimi Atigh, Martin Keller-Ressel, Jeffrey Gu, Serena Yeung@ECCV2022
   https://hyperbolic-representation-learning.readthedocs.io/en/latest/

5. Hyperbolic Graph Representation Learning. Tutorial 2022
   Min Zhou, Menglin Yang, Lujia Pan, Irwin King @ ECML-PKDD 2022

6. Hyperbolic Neural Network. Tutorial 2022
   Nurendra Choudhary, Nikhil Rao, Karthik Subbian, Srinivasan Sengamedu, Chandan Reddy @ KDD 2022

7. Hyperbolic Hyperbolic embeddings in machine learning and deep learning. Tutorial 2020
   Octavian Ganea 2020.

Methods and Models

Hyperbolic Shallow Model

1. Poincaré Embeddings for Learning Hierarchical Representations, NeurIPS 2017
   Maximilian Nickel, Douwe Kiela

2. Learning Continuous Hierarchies in the Lorentz Model of Hyperbolic Geometry, ICML 2018
   Maximilian Nickel, Douwe Kiela

3. Representation Tradeoffs for Hyperbolic Embeddings, ICML 2018
   Frederic Sala, Christopher De Sa, Albert Gu, Christopher Re´

4. Hyperbolic Entailment Cones for Learning Hierarchical Embeddings, ICML 2018
   Octavian-Eugen Ganea, Gary Bécigneul, Thomas Hofmann

5. Lorentzian Distance Learning for Hyperbolic Representations, ICML 2019
   Marc T. Law, Renjie Liao, Jake Snell, Richard S. Zemel

6. Hyperbolic Disk Embeddings for Directed Acyclic Graphs, ICML 2019
   Ryota Suzuki, Ryusuke Takahama, Shun Onoda

Hyperbolic Neural Network

1. Hyperbolic Neural Networks, NeurIPS 2018
   Octavian-Eugen Ganea, Gary Bécigneul, Thomas Hofmann

2. Hyperbolic Attention Networks, ICLR 2019
   Caglar Gulcehre, Misha Denil, Mateusz Malinowski, Ali Razavi, Razvan Pascanu, Karl Moritz Hermann, Peter Battaglia, Victor Bapst, David Raposo, Adam Santoro, Nando de Freitas

3. Continuous Hierarchical Representations with Poincaré Variational Auto-Encoders, NeurIPS 2019
   Emile Mathieu, Charline Le Lan, Chris J. Maddison, Ryota Tomioka, Yee Whye Teh

4. Hyperbolic Neural Network++, ICLR 2021
   Ryohei Shimizu, Yusuke Mukuta, Tatsuya Harada

5. Fully Hyperbolic Neural Networks, ACL 2022
   Weize Chen, Xu Han, Yankai Lin, Hexu Zhao, Zhiyuan Liu, Peng Li, Maosong Sun, Jie Zhou

6. Poincaré ResNet, arxiv 2023
   Max van Spengler, Erwin Berkhout, Pascal Mettes

7. Nested Hyperbolic Spaces for Dimensionality Reduction and Hyperbolic NN Design, CVPR 2022
   Xiran Fan, Chun-Hao Yang, Baba C. Vemuri

8. Hypformer: Exploring Efficient Hyperbolic Transformer Fully in Hyperbolic Space, KDD 2024
   Menglin Yang, Harshit Verma, Delvin Ce Zhang, Jiahong Liu, Irwin King, Rex Ying

Hyperbolic Graph Neural Network

1. [Hyperbolic