<p align="center"> <img alt="DataHub" src="https://raw.githubusercontent.com/datahub-project/static-assets/main/imgs/datahub-logo-color-mark.svg" height="150" /> </p>

DataHub: The Data Discovery Platform for the Modern Data Stack

Built with ❤️ by <img src="https://datahubproject.io/img/acryl-logo-light-mark.png" width="25"/> Acryl Data and <img src="https://datahubproject.io/img/LI-In-Bug.png" width="25"/> LinkedIn

🏠 Hosted DataHub Docs (Courtesy of Acryl Data): datahubproject.io

📣 DataHub Town Hall is the 4th Thursday at 9am US PT of every month - add it to your calendar!

Town-hall Zoom link: zoom.datahubproject.io

Meeting details & past recordings

✨ DataHub Community Highlights:

Read our Monthly Project Updates here.

Bringing The Power Of The DataHub Real-Time Metadata Graph To Everyone At Acryl Data: Data Engineering Podcast

Check out our most-read blog post, DataHub: Popular Metadata Architectures Explained @ LinkedIn Engineering Blog.

Join us on Slack! Ask questions and keep up with the latest announcements.

Introduction

DataHub is an open-source data catalog for the modern data stack. Read about the architectures of different metadata systems and why DataHub excels here. Also read our LinkedIn Engineering blog post, check out our Strata presentation and watch our Crunch Conference Talk. You should also visit DataHub Architecture to get a better understanding of how DataHub is implemented.

Features & Roadmap

Check out DataHub's Features & Roadmap.

Demo and Screenshots

There's a hosted demo environment courtesy of Acryl Data where you can explore DataHub without installing it locally

Quickstart

Please follow the DataHub Quickstart Guide to get a copy of DataHub up & running locally using Docker. As the guide assumes some basic knowledge of Docker, we'd recommend you to go through the "Hello World" example of A Docker Tutorial for Beginners if Docker is completely foreign to you.

Development

If you're looking to build & modify datahub please take a look at our Development Guide.

Source Code and Repositories

datahub-project/datahub: This repository contains the complete source code for DataHub's metadata model, metadata services, integration connectors and the web application.
acryldata/datahub-actions: DataHub Actions is a framework for responding to changes to your DataHub Metadata Graph in real time.
acryldata/datahub-helm: Repository of helm charts for deploying DataHub on a Kubernetes cluster
acryldata/meta-world: A repository to store recipes, custom sources, transformations and other things to make your DataHub experience magical
dbt-impact-action : This repository contains a github action for commenting on your PRs with a summary of the impact of changes within a dbt project
datahub-tools : Additional python tools to interact with the DataHub GraphQL endpoints, built by Notion
business-glossary-sync-action : This repository contains a github action that opens PRs to update your business glossary yaml file.

Releases

See Releases page for more details. We follow the SemVer Specification when versioning the releases and adopt the Keep a Changelog convention for the changelog format.

Contributing

We welcome contributions from the community. Please refer to our Contributing Guidelines for more details. We also have a contrib directory for incubating experimental features.

Community

Join our Slack workspace for discussions and important announcements. You can also find out more about our upcoming town hall meetings and view past recordings.

Adoption

Here are the companies that have officially adopted DataHub. Please feel free to add yours to the list if we missed it.

Select Articles & Talks

See the full list here.

Security Notes

Multi-Component

The DataHub project uses a wide range of code which is responsible for build automation, documentation generation, and include both service (i.e. GMS) and client (i.e. ingestion) components. When evaluating security vulnerabilities in upstream dependencies, it is important to consider which component and how it is used in the project. For example, an upstream javascript library may include a Denial of Service (DoS) vulnerability however when used for generating documentation it does not affect the running of DataHub itself and cannot be used to impact DataHub's service. Similarly, python dependencies for ingestion are part of the DataHub client and are not exposed as a service.

Known False Positives

DataHub's ingestion client does not include credentials in the code repository, python package, or Docker images. Upstream python dependencies may include files that look like credentials and are often misinterpreted as credentials by automated scanners.